Trying to Replicate Processing’s House of Cards code in JavaFX

One of the books I am reading, on and off, is Beautiful data. I recently read Chapter 10 – Building Radiohead’s House of Cards by Aaron Koblin with Valdin Klump and thoroughly enjoyed it. I was mightily impressed that Processing was used in the production of the video, which gave me the inspriation to:

  1. Have a look at the Processing code
  2. See if I can replicate the code in JavaFX!

In the first instance you can find the Processing code, which is part of the Google project, here. I encourage you to check it out, it is really good (especially after reading the chapter in the book!).

Ok, so the code for Processing is really straight forward (I have stripped out comments):

import processing.opengl.*;

int frameCounter =1;

void setup()
{
  size(1024,768, OPENGL);

  strokeWeight(1);
  
}
void draw()
{
  background(0);
  translate(width/2, height/2); 
  translate(-150,-150);
  scale(2);
  
  String[] raw = loadStrings(frameCounter+".csv");

  for(int i = 0; i 2101)
  {
    exit();
    println("done");  
  }
}

With the data in place (1000 CSV files), this works really nicely “straight out of the box”. It is a joy to play with it.

Here is a screen shot of the data represented in Processing:

So, being thoroughly motivated I started work on writing the JavaFX equivalent. In the first instance I needed to load the CSV files. Now Processing has a function for that, for JavaFx I decided to write my own Java class to process it:

public class LoadStrings
{
    URL url;
    FileInputStream fr;
    BufferedReader br;
    List<String> myLines = new ArrayList<String>();
    String a;
    
    public LoadStrings(String fname)
    {
        String theLine;
        try
        {
            url = new URL(fname);
            br = new BufferedReader(new InputStreamReader(url.openStream()));
            while((theLine = br.readLine()) != null)
            {
                myLines.add(theLine);
            }

            br.close();

        }
        catch(IOException e)
        {
            System.out.println(e.toString());
        }
        catch(Exception e)
        {
            System.out.println("It ain't worked!");
            System.out.println(e.toString());
        }
    }

    public String[] getRows()
    {
        String[] res = new String[myLines.size()];
        for (int i = 0; i < myLines.size(); i++)
        {
            res[i] = myLines.get(i);
        }

        return res;
    }
}

With that complete I started on the Main application. Processing’s draw() function is a continuous loop, I believe based on the framerate. With this in mind I needed to replicate the draw function, so I choose to use an animator – Timeline. Before defining this animation I declared:

var lines : Line[] = [];
var theFrame = 0;

I defined my Timeline as:

def projector : Timeline = Timeline
{
    repeatCount: 1000
    framerate: 60

    keyFrames:
    [
        KeyFrame
        {
            time: 0.5s
            action: function()
            {
                delete lines;
                var dataPoints : DataPoint[] = [];

                var f: String = "{__DIR__}data/{theFrame+1}.csv";

                var fr = new LoadStrings(f);

                var rows : String[] = fr.getRows();

                var counter = 0;

                while (counter < rows.size())
                {
                    var thisLine : String[] = rows.get(counter).split(',');

                    var x = Number.parseFloat(thisLine[0]) + (1024/2) + -200;
                    var y = Number.parseFloat(thisLine[1]) + (768/2) + -200;
                    var sr = (Number.parseFloat(thisLine[3]) / (255.0 * 1.1) * (255.0));
                    var sg = (Number.parseFloat(thisLine[3]) / (255.0 * 1.6) * (255.0));

                    insert DataPoint
                    {
                        x: x
                        y: y
                        z: Number.parseFloat(thisLine[2]);
                        intensity: Number.parseFloat(thisLine[3]);
                        scaledRed: sr
                        scaledGreen: sg
                    } into dataPoints;
                    counter++;
                }

                for (dp in dataPoints)
                {
                    insert Line
                    {
                        startX: dp.x
                        endX: dp.x+1
                        startY: dp.y
                        endY: dp.y+1
                        stroke: Color.rgb(dp.scaledRed, dp.scaledGreen, 200,1);
                    } into lines
                }
                theFrame++;
            }
        }
    ]
}

projector.play();

So my stage looks like this:

var theStage : Stage = Stage
{
    title: "House of Cards"
    scene: Scene
    {
        width: 1024
        height: 768
        fill: Color.BLACK;
        content: bind
        [
            lines
        ]
    }
}

It is all very exciting, isn’t it…? Before implementing this code I managed to load the application showing just the first file and it’s data points and it looked exceptional:

I was very excited, although a little apprehensive as the time it took to both compile (over 2 minutes on my machine with the full 1000 files) and running the app. It was a lot slower to load than the Processing instance. I am writing this in Netbeans, on OS X, with 4GB of RAM.

Running this, with time: 0.5s results in a slow animation, which leaves you wanting more. I tried to reduce the time, even commenting it out, but the image couldn’t get updated in time and the effect was even worse.

Conclusion:

Ok, so JavaFX was not really designed for this type of work (well that is my understanding). However I am a little surprised that I can’t run this like I can with Processing. I even tried increasing the memory setting in Netbeans by changing the properties of the project so that Run JVM arguments are: -Xmx3072m.

This is my first attempt at anything like this, so perhaps I am mis-understanding something somewhere.

Perhaps I am misunderstanding animations in JavaFX?

I tried to look into the 3D functionality with JavaFx, but I struggled to find examples I could relate to.

All in all I feel I have tried (and there is never any harm in trying). I like what I have, I have feeling it could be better?

Any pointers / thoughts?

Update: 15th August

So after some excellent feedback from Jonathan Giles, I have changed the code as follows.

The LoadStrings now looks like:

public class LoadStrings
{
    URL url;
    FileInputStream fr;
    BufferedReader br;
    List<String> myLines = new ArrayList<String>();
    List<DataPointJ> dataPoints = new ArrayList<DataPointJ>();   

    public LoadStrings(String fname)
    {
        String theLine;
        try
        {
            url = new URL(fname);
            br = new BufferedReader(new InputStreamReader(url.openStream()));
            while((theLine = br.readLine()) != null)
            {
                myLines.add(theLine);
                String[] t = theLine.split(",");
                dataPoints.add(new DataPointJ(Float.parseFloat(t[0]),Float.parseFloat(t[1]),Float.parseFloat(t[2]),Integer.parseInt(t[3])));
            }

            br.close();

        }
        catch(IOException e)
        {
            System.out.println(e.toString());
        }
        catch(Exception e)
        {
            System.out.println("It ain't worked!");
            System.out.println(e.toString());
        }
    }

    public String[] getRows()
    {
        String[] res = new String[myLines.size()];
        for (int i = 0; i < myLines.size(); i++)
        {
            res[i] = myLines.get(i);
        }

        return res;
    }

    public List<DataPointJ> getDps()
    {
        return dataPoints;
    }
}

The Timeline:

var projector : Timeline = Timeline
{
    repeatCount: 50

    keyFrames:
    [
        KeyFrame
        {
            time: 0.5s
            action: function()
            {
                var f: String = "{__DIR__}data/{theFrame+1}.csv";

                var fr = new LoadStrings(f);

                var dps = fr.getDps();

                lines = for ( i in dps)
                {
		    		Line
                    {
						startX: i.getX();
						endX: i.getX()+1;
                        startY: i.getY();
                        endY: i.getY()+1;
                        stroke: Color.rgb(i.getScaledRed(), i.getScaledGreen(), 200, 1);
                    }
                }
                theFrame++;
            }
        }
    ]
}

projector.play();

This saw improvements, however I could not run the timeline with less than 0.5s. I decided, with the new changes I could load the data into memory. I realsied I would have to do this incrementally, so I decided to define 10 sequences which would each hold 50 instance of the LoadStrings. Then I changed the animation which would resolve each LoadStrings, get the data and build the line. For each 50 it would move onto the next sequence and delete the previous sequence:

function loadDataIntoMemory():Void
{
    for (tf in [0..499])
    {
        var f: String = "{__DIR__}data/{tf+1}.csv";

        var fr = new LoadStrings(f);
        if (tf <50) insert fr into filePointer1;
        if (tf >50 and tf < 101) insert fr into filePointer2;
        if (tf > 100 and tf < 151) insert fr into filePointer3;
        if (tf > 150 and tf < 201) insert fr into filePointer4;
        if (tf > 200 and tf < 251) insert fr into filePointer5;
        if (tf > 250 and tf < 301) insert fr into filePointer6;
        if (tf > 300 and tf < 351) insert fr into filePointer7;
        if (tf > 350 and tf < 401) insert fr into filePointer8;
        if (tf > 400 and tf < 451) insert fr into filePointer9;
        if (tf > 450 and tf < 501) insert fr into filePointer10;

    }
}

The Timeline:

var projector4 : Timeline = Timeline
{
    repeatCount: 500
    framerate: 60

    keyFrames:
    [
        KeyFrame
        {
            action: function()
            {
                var fr;
                if (theFrame < 50) fr = filePointer1[theFrame];
                if (theFrame >= 50 and theFrame < 100)
                {
                    if (sizeof filePointer1 > 0)
                    {
                        delete filePointer1;
                    }
                    fr = filePointer2[theFrame-50];

                }
                if (theFrame >= 100 and theFrame < 150)
                {
                    if (sizeof filePointer2 > 0) delete filePointer2;

                    fr = filePointer3[theFrame-100];
                }
                if (theFrame >= 150 and theFrame < 200)
                {
                    if (sizeof filePointer3 > 0) delete filePointer3;

                    fr = filePointer4[theFrame-150];
                }
                if (theFrame >= 200 and theFrame < 250)
                {
                    if (sizeof filePointer4 > 0) delete filePointer4;

                    fr = filePointer5[theFrame-200];
                }
                if (theFrame >= 250 and theFrame < 300)
                {
                    if (sizeof filePointer5 > 0) delete filePointer5;

                    fr = filePointer6[theFrame-250];
                }
                if (theFrame >= 300 and theFrame < 350)
                {
                    if (sizeof filePointer6 > 0) delete filePointer6;

                    fr = filePointer7[theFrame-300];
                }
                if (theFrame >= 350 and theFrame < 400)
                {
                    if (sizeof filePointer7 > 0) delete filePointer7;

                    fr = filePointer8[theFrame-350];
                }
                if (theFrame >= 400 and theFrame < 450)
                {
                    if (sizeof filePointer8 > 0) delete filePointer8;

                    fr = filePointer9[theFrame-400];
                }
                if (theFrame >= 450 and theFrame < 500)
                {
                    if (sizeof filePointer9 > 0) delete filePointer9;

                    fr = filePointer10[theFrame-450];
                }
                var dps = fr.getDps();

                lines = for ( i in dps)
                {
                    Line
                    {
                        startX: i.getX();
                        endX: i.getX()+1;
                        startY: i.getY();
                        endY: i.getY()+1;
                        stroke: Color.rgb(i.getScaledRed(), i.getScaledGreen(), 200, 1);
                    }
                }

                theFrame++;
            }

        }

    ]
}

This allowed the script to run, for the first time, through 500 files showing the animation. It gobbles up the memory, and I know that this code can be improved, however I am not going to have any time this week so I thought I would post this update for the moment.

Advertisements

Tags: , , , , ,

7 Responses to “Trying to Replicate Processing’s House of Cards code in JavaFX”

  1. Jonathan Giles Says:

    There are a few problems with your code, that once fixed should improve performance dramatically.

    Firstly, I would probably load all of the csv files into memory prior to running the animation (assuming there is enough memory to allow this), or I would run a separate thread that would run ahead of the animation to load the CSV files and also delete them from memory once they were played in the animation.

    Secondly, you’re creating waaaay more sequences than you need to, and this will be very expensive. If I were you, I would replace both loops to create a single sequence as a result of the loop. Refer to my blog post here for more details: http://fxexperience.com/2010/07/sequences-performance-tip/

    With these changes you should see a considerable improvement in speed. I’d be interested to know how the application performs once this is fixed up. Also, these fixes are the low-hanging fruit, I haven’t bothered yet to look in detail at the algorithm and see if it can be improved in general (although, looking at it, I wonder whether creating the DataPoint is necessary, or whether you can just go straight to creating the Lines).

    — Jonathan

    • itssmee Says:

      Hi Jonathan

      Thanks for your comments. I didn’t mention on my posting, but I initially did load the data into memory and then ran the animation. Unfortunately it caused memory error (Java Heap Space), it was also a lot slower to compile and run. So I decided to stay away from that idea.

      Regarding your other points, thank you for that feed back. I have cleaned up the animation function to be:

      var projector : Timeline = Timeline
      {
          repeatCount: 50
      
          keyFrames:
          [
              KeyFrame
              {
                  time: 0.5s
                  action: function()
                  {
                      var f: String = "{__DIR__}data/{theFrame+1}.csv";
      
                      var fr = new LoadStrings(f);
      
                      var dps = fr.getDps();
      
                      lines = for ( i in dps)
                      {
      		    Line
                          {
      			startX: i.getX();
      			endX: i.getX()+1;
                              startY: i.getY();
                              endY: i.getY()+1;
                              stroke: Color.rgb(i.getScaledRed(), i.getScaledGreen(), 200, 1);
                          }
                      }
      
                      theFrame++;
                  }
              }
          ]
      }
      
      projector.play();
      

      This has made an improvement, however I still cannot lower the time below 0.5s.

      Since making this change I have also changed the behaviour of the loadStrings (as you suggested), so it does the work with the data values and returns an ArrayList.

      With all this in place I tried loading the data into memory, this time it causes the application to hang (but at least I don’t get a memory error). There are 1000 files, which totals 500MB of data. The machine has 4GB RAM.

      I am looking into loading the files under a different thread, to see how that helps.

      • Jonathan Giles Says:

        That looks much better, but I do wonder what the getDps function is doing, and whether it is necessary (or whether you can go straight to creating Line objects).

  2. Jonathan Giles Says:

    Also, I would improve the CSV loader class to split the strings at the time of parsing, and not convert the ArrayList to an array in getRows – just return an ArrayList or Iterable.

    This should help speed things up as well, as currently there is a bit too much needless busy-work going on.

  3. JavaFX links of the week, August 16 // JavaFX News, Demos and Insight // FX Experience Says:

    […] has blogged about recreating Radioheads ‘House of Cards’ music video in JavaFX (the original video was created in Processing). For those unsure what is being referred to, check […]

  4. Java desktop links of the week, August 16 | Jonathan Giles Says:

    […] has blogged about recreating Radioheads ‘House of Cards’ music video in JavaFX (the original video was created in Processing). For those unsure what is being referred to, check […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: