stdview.com/sv

Go Frontend Development

This is a research project to find answers into how to develop frontend applications in the Go programming language. We define the following requirements

• MUST be able to write all logic in Go, we can include client side libraries if necessary but its use should not be exposed to the programmer
• MUST have a feeling of that what you type is what will happen.
• MUST Should be about software engineering, people working together.
• MUST Removing the barrier between frontend and backend. Some-one who knows how to program Go should not shy a way from setting up a static website
• MUST should only need to learn one set of tools for the following usecases

Usecase 1) Interactive (mobile) Apps

Some applications require a high amount of custom interactions. Think drag and drop, complicated forms and a lot of data that needs to be loaded dynamically. A special case here would be mobile apps that feel like native apps.

Usecase 2) Dashboard Apps

Interaction is straight forward; the user moves from page to page and mainly fills in forms.

Usecase 3) Static Websites

There is a compilation step that is ran upfront such that a set of static assets (html, css, js, images) can be uploaded and served from a cdn (Netlify, s3 etc).

Usecase 4) Traditional Apps

And finally, there is the model where the page simply gets fetched from the server and the page is reloaded. How the web was initially setup.

Usecase 5) 2,3,4 with Dynamic Elements

As pointed out by the usecase of liveview, sometimes there are websites that just have a few elements of interactive (auto-complete, fuzzy search) etc.

Insights

• 3 and 4 can be served by Go. With the build-in html templating language it is possible to compile to HTML which can be written to a static file or a http response writer.
• 2 can be served by 4 in most cases. If it required that the experience feels more fluid something like https://github.com/turbolinks/turbolinks can be used.
• 1 can currently be served by compling to WASM using official compiler but:
  • this produces large binaries
  • only supported by https://caniuse.com/#feat=wasm
• To reduce the binary size it is possible to use TinyGo, but this is an ambitious tool that misses a lot of the language features.
• 1 can currently be served by gopherjs/joy to javascript compiler. but both of the project have stated to be on hold. Creating another compiler is a moving target as the Go language moves forward.
  • this is the only way to support old browsers with javascript
• Doing 1 for browsers without javascript is only possible with iframe rendering or simply switching to 4.

Research

• RQ1: If we only need a solution for 1, maybe we can use server side state using something like: https://dockyard.com/blog/2018/12/12/phoenix-liveview-interactive-real-time-apps-no-need-to-write-javascript
• RQ2: If the goal is to turn it into software engineering, maybe it is more about building tools to build and test?

Options

• A custom go framework that compiles to WASM: Pro: more future proof, eventually all browsers should support WASM Pro: no server interaction, works in offline mode no problem Con: Browser support of 89%: https://caniuse.com/#feat=wasm but we may include: https://github.com/WebAssembly/binaryen wasm2js to support IE11 Con: currently has 400 KB wasm file, needs to be downloaded only once. 3G download speed is 0.1Mbit - 6Mbit/s (6000 - 100 Kbit/s) (https://www.ofcom.org.uk/about-ofcom/latest/media/media-releases/2014/3g-4g-bb-speeds, https://kenstechtips.com/index.php/download-speeds-2g-3g-and-4g-actual-meaning#2G_3G_4G_5G_Download_Speeds). So downloading 400KB would take 4s in the worst case and 0.067 (67ms) sec in the average case. Chrome 'slow 3g' simulates it at 8.9sec and 'fast 3g' at 2.44 seconds. Alternatively, it can be compiled to TinyGo wasm Pro: potentially binaries in th 30kb/s range, Chrome 'slow 3g' would turn this into 2.54s and 'fast 3g' in 1.22s. So 1.2 second faster on fast 3g
  and 7.4sec faster on fast 3g Con: tinygo is still developing, more focus on microcontrollers and doesn't support a lot of the language features.

• A live view implementation in Go. Pro: All on the server, were go already shines. Needs no extra compilation Pro: will work in browsers that support websockets (94-96%) Pro: Small foodprint, liveview is 23 kb minified. Con: Not possible to create offline experiences Con: lots of roundtrips, latency on 3g will result in poor experience Con: statefull connections, scaling and load balancing will be harder: https://elixirforum.com/t/off-topic-posts-phoenix-liveview-info/18752/8

Plan

A framework in Go, that compiles with official wasm compile. Relieve the downsides by:

• Loading the whole app into one wasm file that is loaded, even if its small elements for usecase 5.
• Make it easy, or by default to show a loading indicator for the app.
• Make it easy or by default to gracefully show that an element is not supported
• Focus on, rendering the html/css fast, and then when the wasm comes in. it becomes interactive. Allow components to render differently, or who a hint to the user otherwise.
• Allow testing of components.

Step 1: View Rendering

First step is creating a DSL that allows defining the View of an application. It should conditionally render certain sub components based on a path slice. In case of static websites this is provided upon rendering, in case off dynamic apps this is passed from the url.location or history api. And in case of server side rendered this is based on the request.URL struct.

Step 2: Routing Design

// links must always be copyable: // - with a hash, for single page apps that don't control routing. The hash // change event may also be captured // - a url that when clicked will go to the server, and either fetch the // index document that is then on the client side rendered to the correct // view? // - a url that is rendered to the correct view on the server side of things // such that almost no changes are necessary on the client side. // - a full url that pushes to the history api and prevents the default // such that rendering happens solely on the client side. // - some a links (not relative?) may not need to be intercepted and should // just work. possibly triggering a round trip to the server // - the server serve should just be able to render a view just based on a path that // is constructed from the request.URL

// Questions // - what about navigation after button Clicks and form submits // - what about url variables?

// Plan // - we always listen to hash changes, they are never send to the server so always // infer client side routing. // - we have server side mux that parses the url path consistently with how any // client side path (hash or otherwise) is parsed // - Link generation should allow for: // - named routes with variables defined, url visual changes should not change the structural code // - generate a hash link to the current document, copying and pasting should // load the same document but move states to that route // - generate a link to another document that when pasted either causes the // server to render it server side, or render the root document. which // when returned to the browser renders the state to the page that was expected // - generate a link to another document that prevents the default but instead // just adjust the history. When copied it should allow the server to either // return the root document or the actual page. If it decides to return the // root document it should go back to the page // - if default behaviour is intercepted and captured for history it should be // possible to turn it off. // - Router is also an attribute renderer that renders routing.href types // and intercept link clicks // - there are build in onClick types that intercept clicks, push history and // emit a routing event instead.

View Definition Syntax

Generated Helpers

• Pro: default go tooling helps with makingit readable
• Pro: no parsing needed, easier to add xss protection
• Con: requires tooling to generate helpers
• Con: adding new/custom elements is hard
• Con: uses a dot import, which is discouraged
• Con: doens't feel very idiomatic
• Con: hard to get used to

String concat helper

• Pro: easily copy examples from web
• Con: fmt can get in the way, might sometimes be awkward to format correctly
• Pro/Con: why not a fully fledged templating laguage then
• Con: easier to create invalid html
• Con: no syntax highlighting for the html

Turnline - TODO

• finish V2.1 ergonomics:

  • come up with routing ergonomics that works on server and client side
  • come up with event decoration ergonomics: - [x] pointers with interface assertions and maybe a way to generate documentation to make it more explicit? Needs a clear guide to explain message must be pointers in that case - [-] types that come with our framework that embeds with a custom type that can be asserted - [-] some reflection (that adds 130kb of payload)
  • come up with client-side ajax ergonomics
  • come up with controlled-forms ergonimcs
  • come up with naming schemes for Msg Types, Route Vars, Decorator Interfaces
    • Message Types not prefixed or suffixed with 'Msg', 'm' or 'Message'. Describe in future tense if it represents something that will or should happened: changeInput. Or pas thense if its something that did happen savedResource. verb should be first. When read in the html it should read natural: onChange="changeInput"
    • Route vars prefixed with route. Then the component/page name it routes to when matched.
    • Message Tags the embeddable concrete type is called after the functionality that it provides: PreventDefault, InputChange and the decorator interface is called after that but with -er, so PreventDefaulter or InputChanger

• Come up with a name with the following requirements:

  • ideas: - line: frontline, baseline, coreline, rootline, spotline, jotline, typeline - arcline, deline - arc - engine, piston, dynamo, turbine - sonic, rapid - pivot, turn - tool - frontfactory - poreline, rifline, riffline - turntower, arcturn, turnline, linepanel, linetool
  • Have a nice two letter acronym that doesn't exist for package name
  • Must be nice to google for tutorials etc
  • Must speak a bit to the aspiring frontend developer, not too goi
  • [-] Has a solid/reliable association
  • github.com/ available
  • No other open source project
  • binary name that works and is unique
  • positions against 'react', 'vue' and 'ember'. Not 'sweet' and 'cute' but reliable and solid. like a foundation, proper
  • sleep on it
  • imagine it being part of the go stdlib, what would the package be?

• Think about basic ergonomics for ajax requests

  • What about commands the generate multiple messages, what about progress?
  • What about errors

• Finish V2.1 Implementation

  • implement view
  • implement engine
  • implement route
  • re-visit routing ergonomics
    • can we not use global state?
    • figure out how to deal with NotFound/Default matching
    • can we make use of the 'tree' construct of existing routing library
    • add routeNotFound that matches everything
  • get isomorphic poc to work, our handler should be able to server as traditional pages and then make the page interactive with our wasm
    • POC for isomorphic routing
      • onClick fires a navigate event, tagged with pervent default(?)
      • renderer is responsible for rendering the navigate with pushHistory
      • test: initial event for DOMRenderer is always navigate
      • test: DOMRenderer: navigate is rendered with an entry in the push history
      • test: DOMRenderer: preventDefault works in preventing the default behaviour
      • test: renderPath should work on both server and browser
      • how to do routing when the whole app is mounted relative to another path
    • POC for (isomorphic) form submissions?
      • Get connected forms to work; handling the "Change/Input" event
      • research the use of reflect with field tags for reading js values
        • PRO: feels elegant and go like
        • PRO: enables reflection in other places (if required?)
        • CON: adds 148KB gzipped to the binary
        • CON: still requires message to be addressable (pointers)
    • POC for isomorphic data loading?
    • test with some assets like favicon.ico
  • make it possible to run in ie11:
    • get wasm2js to work in es6
      • post on golang nuts/ or devs for wasm2js
    • transpile to es5
    • get all the polyfills in
    • test on ie11
  • develop a easy to setup http handler that servers a frontend with the required javascript and wasm included.
  • come up with a more elegant solution for the problem that now requires disabling event bubbling for all events rendered by the DOMRenderer - [ ] maybe the morphdom event hooks can offer something here?

• Come up with core concepts and types of the framework with one line explanations by writing the documentation

• Make it easy to handle error Msg that are passed to the component

• Come up with description. Not "in go" as selling point, but have the same goals as the go language. So it was a food fit.

---

• Finish higher level V2.2 components
  • come up with form ergonics that work on server and client
    // @TODO figure out how to read form values for ajax submission // - validation etc // - for non-controlled forms: invent ref concepts to read values from the dom when needed // - for controlled, make it easy to match change events and update data (map) // - dynamic forms, adding fields on the go // - nested forms
  • come up with data source ergonics that work on client and server

Controlled Forms vs Uncontrolled Forms

Focus on making it simple, safe and Go-like to read complex data from a js.Value. It must be easy to mock/stub, so testing becomes possible. Text selection on input focus is a nice edge case: https://stackoverflow.com/questions/36051883/how-to-select-all-text-in-input-with-reactjs-when-it-focused Something like this would require calling into arbitrary javascript.

• Let a Msg be a reader into the original event with a Tag() that will return the user specified type. The reader allows for arbitrary reading of the original event. There is an implementation on the server side that allows the reader to be mocked with data, maybe even allow for fuzzing.

• Or If Msg implements fromJS(r reader.R) interface, it will be passed the reader to do it its own fetching from the event. This better decouples interop code from update logic. But prevents reading different event info based on the state, but this is probably discouraged anyway.

• Forgo field tags for unmarshalling from js values. Will never be as powerfull or explicit as actual code and add a lot of size to the binary. Field tags may be added later, or optionally if the user wishes it. Also don't use decorators, for every type of event. It is too hard to predict how even reading will be use to think we can write decorators for all.

• Refs are also js readers such that it allows for dom interop as an escape hatch Refs need to work on the server as well and allow for testing. Refs need to allow for calling .focus(), select(), play() on elements. But not necessary setting properties. We want to protect the user from using this to shape the dom manually.

TODO:

• research if we can make a fromJS() interface without requiring the Msg value to be a pointer: yes, if the user im plments the interface to return a Msg type value that incorporates values

• research and POC a ref attribute on the view, how it interopts with the component state.

• figure out ergonomics of fromJS() and refs and Readers. Can refs be non pointer types such that the zero value is usefull?

• create poc ref implementation that works with old view definitions

• if we're doing a major refactor to the renderer anyway, see if we can replace the hack that prevents all bubbling

• and allow both controlled and uncontrolled form inputs

• and maybe make it possible to use value attribute on mutiple selects and textarea

• maybe factor away the strconv package

• BUG: if the root element of a component is not the same tag as the component view return tag, handlers will not work.

• MUST be able to ergonomically read values from the original event because it must be possible to develop components that respond to mouse events and need to read x,y coordinates for example.

• MUST be able to read values from a DOM element (refs) ergonomically. Because it is required to read for multiple select, text selection and other usecases documented here: https://reactjs.org/docs/refs-and-the-dom.html#when-to-use-refs

• MUST be able to do controlled and uncontrolled form inputs. Controlled is more powerfull but uncontrolled is simpler and allows for better interop with 3th party libraries. An excellent article about the differenc is here: https://goshakkk.name/controlled-vs-uncontrolled-inputs-react/

• MAY getting and settting value for (multiple) select inputs is awkward. Would be nice to streamline this for the user so multiple select can easily be used with controlled componen workflows.

• MUST develop a javascript interop reader interface that makes it easy and safe to work with values from javascript. First off for refs, but might be handy for other uses as well.

Design Iteration into Forms:

For controlled forms: allow message to implement a interface that allows it to read the raw JS event and return a copy of its type to be passed to the update function. This isolates the logic with the event, allowing it to be re-used between components if necessary. It also doesn't require the creation of pointer types.

For uncontrolled components the rendered should be able to update the ref of a component as a new element is rendered. The ref is a pointer to a js value by (almost by definition) so it would be initialized in the constructor of a component. The createRef rfc describes the need for a simple ref API: https://github.com/reactjs/rfcs/blob/master/text/0017-new-create-ref.md but the callback API is still more powerfull as it allows for setup during component creation and destruction: "React will call the ref callback with the DOM element when the component mounts, and call it with null when it unmounts."

In a functional world operations on refs would be called side effects. How can we make these interactions testable? Well isolated? How to make them composable? How about on server side code? What does NewRef() do there? Can we pass a mock?