An Experiment in application composition by iFrame

This isn't a super serious blog post, but I wanted to share something I haven't seen too many examples of. It's an idea I've had kicking around in my head for sometime. That being composing applications up from iframes and handling communication between them and the host using Post Message. By all means I'm sure it has been done ¹ before but I don't see it talked about all that much ².

The backdrop to this

I went back to university at the start of the year, and in the first term a few classes we would talk about concepts in terms of models which can often be expressed in mathematical terms, often depicted as graphs. I personally just found it hard to digest some of these, and I really wanted some kind of point of reference to think about these. We'd also often repeat the same computations and calculations, over and over which is something as a programmer is easy to solve.

But after implementing some of these the way many of these things interacted became more apparent and easier to remember. So in the following term I decided I'd do just that for most of my classes.

Note book runtime

So in the following term I decided to make an app, which in my words allowed you to create a notebook. Not necessarily in the Jupyter sense, possibly more so like a "Storybook" sense with a bunch of tabs on the side for different stories, with a bunch of knobs on the right hand side to tweak things, but each notebook largely existed to document my notes on a topic. Here are some examples:

• Complex Analysis, Complex Addition and multication
• Micro economics, Demand Curve
• Micro economics, Trade
• Micro economics, Monopolies

Anyways a few of these ³ as a means testing my understanding of these topics and how the interacted with related concepts. On top of that it was also just fun to make. I think there's a lot I'd like to say about different things I've tried in this project but for now I'll stick to the topic at hand iFrames. But I guess briefly before that I'll talk about the motivation that led to using them.

Canvas2D wasn't always cutting it

During the previous terms there were some occassions I thought some things could be expressed with a 3D plot, I managed to do something like that in matplotlib in python with some data I was working with in python. This was in the term before I started on the notebook in a statisics class (there are 3 terms a year at the university I'm at). And in the second term there were times I wanted to model something in 3 dimensions (such as supply elasiticity in a supply curve). So between breaks I decided to dabble in WebGL and try the WebGL fundermentals tutorial for the 10th time ⁴.

Dabbling in WebGL

So over the break between the last and current term, I made this as I went through the various tutorials on "WebGL fundermentals". It was fun. I was hoping by the end I would be able to add some 3d graphics to my notebook this coming semster. Although in order to do that, I needed to figure out how I was going to encorporate it in the runtime of the notebook.

Adding WebGL to the notebook

This is a whole topic in of itself, but I didn't use any libraries for the notebook (it felt more fun that way), so gradually through the process of adding more and more content to this, I gradually made an notebook/application runtime that allowed me to define portions of a notebook as seperate widgets that allowed each notebook to specify it's config declaratively, and each portion of content was a "widget".

Anyways at this point all I had at that stage was a document widget type and a canvas2d widget type. So I guess I needed to make a WebGL widget type? However if I was going to do that I needed to decide how these widgets interface with the runtime and how I want to handle stuff like initialising buffers, shaders, etc. Chances are it will be a pain in the ass to update the more its used. So I just put it off till the term started. Eventually a usecase came along...

We looked at the cobbdoulgas production function in the first week of marco 2, which is the perfect candidate seeing as there are multiple inputs to this, there's actually 3 (technology, labour and capital), but I decided to simply control the value of technology by a knob. You can open it here and it might be easier to modify controls there, I've placed it below regardless. Now this is where we start to talk about iframes.

I spy with my little eye
a HTML tag beginning with i

After making the above example as part of my WebGL sandbox page I was wondering how to replicate it within my notebook. At this point I was like, I guess I can just embed it? So I decided to add a widget type for iframes, and it worked. On the iframe side I added some url params to disable some of the UI elements, but I hadn't quite decided how it ought to keep it in sync with the request of the graphics on the page. Like I knew I would use WebGL I just wasn't sure what kind of interface should exist between the iframe and the notebook.

In the end I decided each iframe widget should tell the runtime how to shape the messages, as I couldn't really count on the iframe complying with an interface I define in my notebook, and if I decided to modify that interface I would need to update all these seperate iframes which may or may not be part of the same code base (at the moment, this webgl widget is not).

And before moving on to how the message is received this is how the runtime interacts with the above code before sending the message to the iframe, infact here is the source for the iframe wiget.

The child side of the iframe

I'm not entirely sure what the most appropiate term to use here to refer to the embeded side of the app, but I guess child is correct...

In order for the iframe to respond to the messages sent on postMessage the child side needed to be updated to recieve those updates. I still wanted to be able to use this WebGL app independently of my notebook using the existing knobs on the child side, so I decided to abstract the wiring of the inputs on to the UI out of that code, making it more general and allowing it to work for both setting up the knobs as well as receiving messages from a parent window. But here's how the technology and alpha knobs were setup.

Next there's this (note I'm not sure why I called the constructor that, but nevertheless below is the constructor function for the config source).

And lastly here's the different implementations of the different configuration sources.

The end result!

After all that, now we can communicate with the iframe from the notebook!

I'm not sure what I'll do from here, but I'm sure all of this will gradually evolve over time. Anyways I just wanted to post somewhere about my experience working on it so far. I think something like this might be good for functionality that is otherwise awkward to encorporate into the notebook source without modifying the abstraction boundary between the widgets and the runtime before having a chance to think about how I want to handle that. More so extensions I haven't given sufficient thought on how I'd like to approach. Because once I add it'll only become harder to change overtime.

Things I'd like to try in future

There's a few ideas that I'd like to try, however for most of these they could probably be implemented in the system itself, so the main benefit of making them embededable is that I can reuse them. Which would be nice if I've gone to the effort of making a complex tool that simplifies a nontrival task.

• Data Analysis: One idea I had was a widget type for loading large amounts of data and running some statistical analysis on it and providing it back to the host notebook (seems like a fun candidate for learning WebGPU).
  
  Performance niches of iframes

  Loading an entirely seperate app inside another application is expensive, as you're effectly a seperate tab, some runtimes even make the two tabs run syncronously between each other, which isn't great. However this is also true for any chunk split code (perhaps minus web workers).

  Based on what I've seen, what you can't do with an chunk split application is unload any chunk after it's been used. Once a script is been loaded by a js runtime it won't unload it from memory.

  That said this the above use case has more to do with large amounts of data than large amounts of javascript being loaded.

• Semantic Formular Editor: I'd like something to modify formulars and output them as a syntax tree, while also having the option to output derivatives and intergrals of the same formulars. Maybe I can expose series of data to it or provide exogenous variables to it based on application state?
• Matrix Editor: Similar to the above, possibly even just a subfunction of it.
• Digram Renderer: There's been a few occasions where I wanted add a diagram to my notes, and which I could just specify it declaratively without worrying about layout..
• The notebook as a widget coordinator: At some point I'll make so iframes output information based on user input or some data processing, which will end up getting stored in the notebook state, which other widgets can read from (like the above cobb douglas widget). This may mean the notebook ends up being some kind of coordination layer for these different widgets, which means this notebook runtime could act as a means to quickly make a data tool, or dashboards or a means to build an application from several smaller ones which were designed from the start to potentially anticipate external inputs and outputs.

Most if not all of these could probably be implemented in the existing notebook runtime, but I guess one thing I think is kind of fun with an iframe is you kind of get to start with a blank slate without having to worry about integrating the functionality with the rest of the system.