Making Highcharts Series from ActiveRecord items

Following up on the previous blog, where we successfully linked categories on the X-Axis with an Active Record table, we'll move on to the next step, that is actually taking data to be used and displayed on the Y-Axis.

But first, a few things:

The method I described the other day to iterate through all ethercraft names in our controller is actually not necessary. Someone (thank you!) pointed out to me that a method does exactly what I was doing already: .pluck.

So instead of above, in the controller action where this method was called I just did:


@ethercrafts = Ethercraft.pluck(:name)

And I can safely remove this method and all.

Next, we'll need to make sure we can actually get data for our series. In the previous example, we used a census to obtain the number of people practicing a certain ethercraft by country. So, we'd need to make sure that we can link countries generated via ActiveRecord in case we add/remove more countries.

Let's change this a little (because I already have Race set up as a selection for characters) and make our graph just draw a census of number of ethercraft practitioners by race. We just need to make a race table, along with the associated actions to add/edit races in the app with the actions, and it'll be fairly similar and ready like ethercrafts.

Now, the challenge next is finding the data we need to actually add the series and have actual values for the graph itself.

If you recall the format for generating series in highcharts:

The series itself is expecting:

•  An array, that contains
•  hashes for each series, with each containing key/value pairs of
• a name for the series, and
• data, which contains
• an array that contains counts for each category on the graph

Keeping that in mind as we make our ethercrafts-by-race census, we'd need to have a hash for each 
race that has data for how many of that race practice an ethercraft.

But since we live in an era of databases and charts and linked information pulled from repositories, we'll generate that data ourself.

In this sandbox app, a fantasy character generator, I've got characters, and each character has a race, and an ethercraft. To make this data count work, I'll link characters to their race and ethercraft choice in ActiveRecord via foreign keys. Each character belongs_to one race and belongs_to an ethercraft.

Once that was done, the next step was very simple: to generate the series, we'll just need to add to our controller a new method that'll iterate through characters, iterating each ethercraft for each race.

This method, defined in the controller will take all races and ethercrafts and iterate through each ethercraft for each race, finding characters that meet the current race/ethercraft combination. The glory of map is that, when used in this way, it generates an array the way we want it to. It creates an array for the counts for ethercrafts in order of their id, and fixes them into the race_ethercraft array for each race. At the end of the ethercraft map, it makes a hash taking the current race's name and the array generated for a data key, then that is automatically added into an array for the results for mapping the races. Once that's all done, we'll have every race with a hash that has the numbers we're looking for.

Now all that's left is to fix up our HTML and our AJAX file:

AJAX:

And we'll boot up our chart again:

Presto! Of course, this was made after I quickly rigged up a bunch of characters in the database, and even then - it still looks empty. But if you had real data to add from a database, this would be one way to do it.

The issue arises, that I can think of is in how that method was written: assuming I had a lot of data to go through, would the chart load as quickly? One way to find out is to actually get a lot of stock data. And that's something I'll look into for next time.

Hope this helps, and as always, comments and critiques are appreciated!

Passing raw array variables from Rails to JS to HTML

I was working on highcharts to follow my previous blogpost, and I got the chart to work. That's great, but highcharts is incredibly flexible and has a lot of features one can use to create dynamic and interesting charts.

But to actually get there, I need some data beyond static example data of John and Jane eating fruits. So, I worked on passing PSQL database objects into the AJAX file in Rails, so that I could apply the variables I want into the chart. How and why would someone do this? It's fairly obvious why, but an example would illustrate things a bit better.

The chart is merely an example; there's many situations where one would want to transfer a database item, or value, or anything into their AJAX file, but that's moving things from Ruby to ERB to JS to HTML. The core issue to keep in mind is to keep the object being passed around readable all the way.

In this case, we'll make a chart that depicts a census of people with different jobs in different countries. In this fantasy fiction world, we'll call them ethercrafts to describe how they cast their magic spells in their day to day. There are 2 countries in this example, Zahnheim and Rogalia. The end result is to look like this:

There's 5 different ethercrafts in the chart: Soulreaving, Enforcing, Invoking, Evoking and Empty. The bar graph (admittedly, not the best chart type to describe population, but it'll suit this blogpost) will compare the two countries' populations for each type of ethercraft.

For this post, we'll focus on the X-Axis descriptors, the ethercrafts. Currently, it describes those by the following code in the Highcharts javascript in the ajax file:

Going down the script:

The first variable, ethercrafts is a list of the ethercrafts as strings in an array which will be the xAxis categories, (we'll talk about this further down). Next is an array to describe the countries that will be the series in the graph. Each country is listed, along with the population of people with a certain ethercraft described in magic constant numbers. We don't want magic numbers, but for the time being, those will work for this example - country population variables will come next time as I make progress on highcharts.

Next variable is the options that we'll pass into the chart we want to make. Highcharts lets you name this anything you want, as this variable will just contain a lot of data which will be passed in as an argument for the Chart function which expects its options. The options include where it's being rendered to (in this case, an HTML object with the ID 'container') and the type of chart, which is bar, the title of the chart, what the categories in the xAxis is (which is our ethercrafts variable, where it expects an array), and the yAxis, where we describe the numbers as population in thousands. Lastly, we describe the series - which expects an array, along with the relevant data. It passes in the countries variable from before.

Lastly, we have the last variable which is a function to create a new highcharts chart with our options passed in as the first argument.

We'll focus on the ethercraft variable here. Right now, this is what we have for ethercrafts in the AJAX file:

But these are all constants. What if (and we have) an Ethercraft class in our ActiveRecord, which also describes these 5 currently and would like to use that? Imagine if there's more than just 5 ethercrafts, like how there's more than 5 different genres of music and we wanted the graph to reflect that?

We'll first need to get all the ethercrafts in ActiveRecord first, and present it as an array of the names of Ethercrafts. Otherwise, we'll just have an ActiveRecord Relation object that describes all the items or the Object References to the Ethercraft items, or worse.

Firstly, in the controller where the graph will be displayed, we can use the following private method and store it in a single instance variable, such as @ethercrafts:

With that done, next we'll need to change the variable in our AJAX file to take in the collection of the different types of Ethercrafts from ActiveRecord (via our instance variable, @ethercrafts). To do that, we can go to our application.html.erb, and add in our header:

Then we should change the ethercrafts reference in the AJAX file and point it to window.ETHERCRAFT. And this should do it! Right?

...no? It doesn't work, because the syntax for performing ERB is expecting quotation marks around that. So we'll go do that:

And this will make:

...what on earth? Oh, because it expects an array, right? So adding a bracket surrounding the variable should close it up nicely:

This should do it:

 Not exactly what we expected. We got the names of the ethercrafts all there/ But they're covered in " before and after!

What exactly happened here? The long and the short of it is that during the processing via Ruby/ActiveRecord -> ERB -> Javascript -> HTML, the array of our ethercrafts were in this form:

["Soulreaving", "Enforcement", "Invoking", "Evoking", "Empty"]

However, through all that, the quotation marks were read and parsed into the " text. So how do we fix this? Let's take a look at the source code while we're at it (very important thing to do as well, it's a good habit I've picked up.).

Our window.ETHERCRAFT has our ethercraft array within the array we added earlier. That's strange! However, this does explain some things. When JavaScript expects an array of strings, like when we are trying to define the X-Axis Categories for Highcharts expects quoted strings separated by commas. Now, the quotes don't exist when Javascript reads it, so we end up with the " mess or a plain syntax error prior to adding our overarching quotes or array boxes. But now, we end up with this.

So how do we fix it? A handy function to use is to use the raw form of our HTML to be read into the Javascript, which we'll add to our ERB file as we render our ethercrafts variable:

We remove the quotations and the array, as our instance variable already contains our square array and our strings separated by quotes and covered with individual quotation marks for each through our method in the controller. This is exactly the format Javascript will expect, so this is what we get:

Hooray! We've done it.

And so ends this post where I describe my morning. I get closer to making highcharts usable for census data, and learn how to move raw data from Rails to ERB to JS to HTML. A very nifty lesson, overall.

Hope this helps. Comments, criticisms and advice are wholly welcome!

Starting Highcharts

I was looking into highcharts due to the popularity of presenting data on websites and apps nowadays. Highcharts is part of a series of JavaScript based apps that features interactive display along with HighStock and HighMaps - and they look pretty sleek.

As you highlight your mouse over the bars, a small box describes exactly what you want. And the implementation thus far seems to be so simple. I'll be working on this over the next few days to see how I can put in variables as data - the site has a way for people to get started.

But, keep in mind that on Ruby on Rails we like to compartmentalize our scripts and sources. Normally, you'd put in your head tags  (as instructed in the Highcharts tutorial:)


<'script' src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></'script'>
<'script' src="http://code.highcharts.com/highcharts.js"> </'script'> 

But there is now a gem for highcharts, highcharts-rails. So all we'd need is to add this gem to our gemfile and we can just plug in highchart scripts and methods into our Ajax file:


In Gemfile:
gem "highcharts-rails", "~> 3.0.0"

In app/assets/javascripts/application.js, add to the end:
//= require highcharts
//= require highcharts/highcharts-more

And then we're all set. This above example was as easy as making a new div with the id 'container' anywhere on my HTML, and using the following, copy pasted code from the highcharts example into your JQuery code:

From the look of things, we could probably put in variables in the array. This is looking like straightforward JQuery. I will definitely be meddling around here to get what I want for statistics. It definitely seems like a versatile and very powerful chart API.

The little things between form and 'form'

After a long trip following my graduation from Metis, I went back proper into a routine this week. And today, I learned a great tidbit about how to render forms in Rails, very simple but it saves me a lot of time from now on.

I used to write the following:


<%= render 'form', form: form %>

For rendering a form partial. I couldn't just not put in the 'form: form' part because it would always say that form doesn't exist in the partial (in the form partial, there was - obviously, a form with it's associated fields designated as form such as form.text_area, and so on).

However, one of my colleagues pointed this out to me that I could just do:

<%= render form %>

The difference? By removing the quotations, Rails magic will immediately do the previous line I put up for me. That is very convenient, and actually solves a lot of my hassles (I've always wondered how people just write render form - and now I realize it's just the quotes!).
 

Using Carrierwave and Fog

Following up installing RMagick in my previous post, I added Carrierwave and Fog. For the most part, instructions on getting Carrierwave running with Fog are in line with what I gathered in this blog post by JavaHabit. However, there's a few things to keep in mind:

• Once you get your fog.rb generated, be sure to move it to your config/initializers folder.
• If you plan on pushing your app to heroku, don't link your secret keys within the fog.rb file, you should install the dotenv-rails gem, make a .env file, put the keys inside it, then be sure to use ENV.fetch in the fog.rb file to obtain the keys remotely. Add .env to the gitignore file.
• Restart your server. Additionally, it's best to convert the hash syntax within the fog.rb file to the newer rails style (if you use version 4).
• Common error fixes: restart the server, remove the region line, and keep the fog.rb file to the bare minimum (delete mostlines to make most things set to their defaults). Those are the most common SO answers I've found.

Basically:

Add the dotenv-rails gem in your gemfile:

gem 'dotenv-rails'

In your fog.rb file, trim it down as much as possible and make the key hashes point to an ENV.fetch of the keys you need:

CarrierWave.configure do |config| 

config.fog_credentials = {
 provider: 'AWS',
 aws_access_key_id: ENV.fetch("AWS_ACCESS_KEY_ID"),
 aws_secret_access_key: ENV.fetch("AWS_SECRET_ACCESS_KEY"),
 }
 config.fog_directory = "bucketname"
 config.fog_attributes = {"Cache-Control'=>'max-age=315576000"}
end

In your .env file, make the following definitions for env variables:

AWS_ACCESS_KEY_ID=accesskey 
AWS_SECRET_ACCESS_KEY=secretaccesskey

In your gitignore file, add the uploads folder and .env to avoid uploading your images (if any) and secret keys onto github:

/public/uploads
.env

Lastly, in ImageUploader:

storage :fog

And that should set Fog up for your Carrierwave. Hope it helps!

Installing RMagick on Windows

Today while working on our project, I was facing trouble installing RMagick. It was - as expected, a windows issue, and while most guides on troubleshooting were old and out of date, a more recent Stack Overflow response (January 2014) more or less pointed out the issue was that Windows might not be supported in the more recent versions of ImageMagick with RMagick. So the solution was to use older versions when it worked, and it definitely solved my issue in installing it.

To sum up and quote the answer by Dany Khalife:

Uninstall ImageMagick and re-install a previous version that works with your Ruby setup.
This is what worked for me :

• Ruby 1.9.3
• Dev Kit 32-4.5.2
• ImageMagick 6.7.7 (i had a really hard time finding binary versions for legacy versions of ImageMagick so i guess this link will be handy for you: http://ftp.sunet.se/pub/multimedia/graphics/ImageMagick/binaries/)
• Rmagick 2.13.2

Keep in mind the version numbers! I'm running on Ruby 2.0.0 and it worked just fine. Be sure to use the following command, assuming ImageMagick was installed in C:/ :


gem install rmagick --version=2.13.2 --platform=ruby -- --with-opt-lib=c:/ImageMagick/lib --with-opt-include=c:/ImageMagick/include 

And that's all there is to it. 'bundle install' should work just as well, too for the rails project. Hope this helps!

A month of Metis

By the end of today, it will have been 4 weeks since I started learning at Metis. There's a lot that I've learned, and it's impressive when I think of how now I have the fundamentals to already build something great.

If there's something that really draws me into programming, it's that sense of creation you have at your fingertips. You can code together magnificent pieces of software, like how an artisan handles his craft - this is the builder of the tech era.

At Metis, broadly the four weeks covered the following:

1. HTML/CSS
2. Ruby
3. PostgreSQL
4. Sinatra, irb
5. Ruby on Rails

Most importantly, I feel are not just knowing the tools that we have at our disposal, but also the fundamentals behind these tools, and practices that help us to be more effective and efficient. We've had to build from ground up methods in Ruby, Sinatra and Rails that are so easily simplified into one-line commands, or we've had periods of free time to come up with our own code and write a solution, and get what we want in our apps.

One huge draw is the fact that programming is like a game, or a puzzle. There's many ways to solve these puzzles, and you learn that some ways are faster than others, while others are more effective or clean to look at. There's a bit of joy when I go back to old code I've written, see what I've done, grin and rewrite what I could've done better now that I'm a month into the study.

I wrote some code for the zombie dictionary a few weeks back, now, and maybe when I go back to it it'll be significantly different by the end.

That said, over the weekends I'm slowly working on getting to know Vim, and while it more or less functions similarly between Unix and Windows systems, the fundamental differences between them will take me some time to adjust - for example, we utilize Homebrew, Vundle, Thoughtbot's RCM, and dotfiles to handle and manage packages, bundles and configuration settings for Vim. With the exception of Vundle which has its own set of installations for Windows, everything else is made for a UNIX system in mind. However, I'm learning of several softwares that emulate similarly. For homebrew, I've learned of chocolatey ; and quick searches for dotfile installers and bundlers for windows have turned up a few that I still need to go through.

Given this, an important thing that I learned is the core differences in how OSX handles its system-wide installations compared to Windows (which uses environmental variables to set PATHs), and the inner workings of the dotfiles we've been installing. Ultimately, it helps me understand what is being changed 'under the hood' when we brew a bundle or run RCM. (Most of the dotfiles change internal settings in Vim, Git, RailsVim and TMux among many others).

Like I said, there will be a part two for that, so that should be coming following further dives into converting from Sublime Text to Vim.

Rails and Resources, Part 2

The all-important table! It's crucial this week that we have most of this table down, and honestly, I wish I wrote this blog post before our weekly quiz. Definitely would've helped!

Basically, each action here is a method defined in the table_controller.rb file (in this case, wombats). The paths are defined in the routes.rb file (under resources :wombats) in their most default state. Each path functions under a particular HTTP verb protocol. Using rails, we don't have to fully type a link in HTTP, but we can use rails code (link_to or form_for) to quickly make the connections between the records within the database (via the controller, that takes in the instances of said models with the corresponding tables based on the action used) and back within our html.erb file.The instance variable objects are basically stored references to particular database segments (such as the records we're looking for in Class.find(:id)), so when we link_to the instance variable, Rails already knows what and where we're referring to.

I know one of our classmates is working on a blogpost to fully describe the actions, so I'll be leaving this blogpost short and link to hers when she's finished.

Rails and Resources, Part 1

This week, we began studying Rails. Compared to last week, I didn't run into any bumps between UNIX/Window differences thus far, so that was a relief. My PSQL is set up correctly (though I've yet to dig through security allowances and authentication more than I needed to - I'm almost sure that we'll be going through that, though), and I'm able to install rails with some ease.

This is probably the part that I had a few hiccups compared to the easy installs the UNIX terminals had, but this was all just because I didn't have all the installations ready.

Basically, I installed ruby, but not particular gems and additional devkits that were necessary to set things up for installing rails. Luckily, when I tried to install Rails, the error messages were -very- helpful (it's important to read them!) in pointing me at what I needed to do next. First, I needed to install JSON via gems, but then when I was told I couldn't install JSON via gems, they pointed me to make sure I had the devkit in my PATH if I installed it already. I don't think I installed the ruby devkit.

First things first, I needed to get the Ruby Devkit 's latest version. Once that was done, I ran the gem install command for the JSON version I needed (ruby gems will tell me what the exact version and command I needed once I tried to install rails on failure). After that, it was a simple matter of using 'gem install rails' and I was set.

Rails is fairly easy to comprehend, given that we spent the last week working with the stuff 'under the hood' - it makes all the things we did in Sinatra and PSQL so simple and straightforward, and there's lots of ways to shorten the process of pulling database entries onto a website.

Two of the most crucial things we had to consider however, was how the entire Rails system worked from request to response, and the other was the Seven Route Resources (tm?).

First, the Rails system from Request to Response: assuming the database is already made:

From the Browser, we make a Request to the App Server. It receives the path specified in the request, and properly routes it through a controller. The controller may instantiate the Model, which allows it to pull, change and request information from the database. From there, it renders the view based on what information it has and has been given, which becomes the response sent through the App Server and back to the Browser which displays the response, likely as an HTML file.

Browser refers to the user, or the browser who is navigating the html page and making the request.
The command 'rails s' boots up the App Server.

Routes refers to the routes.rb. It takes the Path of the url and accordingly performs the correct action via the controller.

Controller refers to the tablename_controller.rb. This is where the various actions are defined (akin to methods). What it exactly does depends on what Route is taken in the previous step. Primarily, it will set instance variables for the Model and View to use. It can pull/send database records/changes on request based on the Route via the Model, and render the View with the information it has before sending the response to the request back to the Browser. In the case of a redirect (HTTP code 3xx), it will not use Model or View and return the redirect response to the browser, which will initiate a new request to render the target URL.

Models refers to the modelname.rb file. When it gets instantiated, it's akin to using "tablename = Class.new". As classes in Rails in this case inherit ActiveRecord::Base, the object 'tablename' here is of type ActiveRecord and has access to the specific commands we use in Rails. Depending on what the controller requests, it may pull some information (.find, .where), pull all the information (.all), create a new entry (.new, .create) with the associated database table (which the Model is named after).

View refers to the .html.erb file. This is where the html.erb file containing all the text, code, and variable references are laid out. The Controller will read and render when it requests for it.

DB refers to the database.



====
Browser --request--> App Server --> [Rails Stack: Route --> read path --> Controller --> Proceed based on path:
====
Controller <--instantiates and requests based on routed path--> Model <--pull, change, create entries--> DB

AND/OR

Controller -- Reads based on route -->View --Rendered --> Controller --> App Server --returns response --> Browser

OR

Controller -- Receives redirect route --> App Server --returns response --> Browser
 ====

That sums up the entire process. Hopefully, this will help others understand as well. Thanks to the instructors for being patient and making this clear. Next part, I'll describe the Actions/Paths associated in the Resources in Rails.