oribuild-darwin-amd64

ori

Tools and plugins to run innerloop builds of typescript monorepos using esbuild.

Configuration & Usage

ori -h

  -blockFollowUp
        Wait for an initial build before running non-build tasks (implied by -traceInitialBuild)
  -cacheNodeModules
        Forcefully cache node_modules in a plugion
  -config string
        Path to build.json (default "./build.json")
  -cpuprofile string
        Generate a cpu profile at the given path
  -entry string
        Use a given entry or entry group (specified in build.json)
  -findWithLs
        defer trace collection until the incremental rebuild
  -gitRef string
        initial set of changed files to use when starting the typescript process (default "HEAD")
  -logLevel string
        log level (error,info,debug) (default "info")
  -noTui
        Disable the tui and print everything to stdio
  -port int
        Port to run the http server on (default 3000)
  -snoop
        snoop on the import array
  -split
        Enable codesplitting (WARNING: Requires type='module' on the entrypoint script tag, will not work against prod owa-web-server)
  -trace string
        Generate an event trace at the given path
  -traceIncrementalBuilds
        Collect a pprof trace of incremental builds
  -traceInitialBuild
        Collect a pprof trace of the initial build

build.json fields

{
    // Where to find a partial tsconfig with paths: entries
    // mapping requires from lib directories to the corresponding
    // source files
    //
    // should be deprecated by #3
    "tsconfigPathsPath": "tsconfig.paths.json",
    // The path that esbuild should output to
    "outPath": "dist/esbuild",
    // Where to find resource.json files
    // 
    // TODO: document resource.json files
    //
    // Should be deprecated by #4
    "resourceRoots": ["packages", "shared"],
    // Where to find source files to watch
    // 
    // Should be deprecated by #4
    "watchSourceRoots": ["packages", "shared"],
    // Directories the webserver should serve in addition to serving
    // resources from resources.json and the built scripts + chunks
    "directServeDirectories": ["resources"],
    // constants to define, passed to esbuild's define property
    // see https://esbuild.github.io/api/#define
    "defineConstants": {
        "global": "self",
        "process.env.IS_WEBPACK": "false"
    },
    // A map of entry script names to the packages they are built from
    // (packages are read from tsconfig.paths.json, should be replaced
    // with packageNames in the workspaces we crawl)
    "entry": {
        "mailindex": "mail-index-package-name",
    },
    // Entrypoints to workers
    // 
    // Workers are built separately, see WorkerLoader for details
    //
    // Worker entrypoints not in this map will be built inline in the main,
    // build, at significant performance cost
    "workerRawEntries": {
        "pdfjsworker": "node_modules/pdfjs-dist/build/pdf.worker.js",
        "pdfjsworkermin": "node_modules/pdfjs-dist/build/pdf.worker.min.js",
        "owadataworker": "packages/libraries/worker/owa-data-worker-bootstrap/src/index.worker.ts"
    },
    // Human readable groups of entries from the above entries map
    // as well as custom extensions to defineConstants
    //
    // For use on the cli for common entry goups.
    "entryGroups": {
        "OWA Mail": {
            "entries": ["mailindex"],
            "defineConstants": {
                "OWA_BUILD_CONSTANTS.ENTRIES.mail": "true",
                "OWA_BUILD_CONSTANTS.BUILD_ALL": "false"
            }
        }
    }
}

Working on ori

Getting Dependencies

• install go 1.17 https://go.dev/doc/install
• install mingw-gcc. I do this via https://www.mingw-w64.org/downloads/#mingw-builds (the sourceforge link at present)
  • If the live installer fails, you can download the prebuilt binaries directly and add those to your path https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/
  • Prebuilt binaries require 7zip to extract https://www.7-zip.org/
  • I installed x86_64-posix-seh This is to support building libsass https://github.com/wellington/go-libsass/issues/37
• get the install path of mingw-gcc's bin and add it to your path (in my case /c/Program Files/mingw-w64/x86_64-8.1.0-posix-seh-rt_v6-rev0/mingw64/bin)
• Install the golang vscode plugin, and click "Install All" when it prompts you to install missing golang components (godef, gopkgs, gopls)

Catches

Add more here as you hit unexpected situations

• in client-web: yarn gulp gqlgen:generate needs to be run manually after any graphql change.

• node_modules are not monitored and assumed to be always stable. If you edit node_modules, you will need to save another file to refresh. Once separate builds are implemented (#8), you will have to restart the whole build agent, unless you specifically omit that node_module from the build cache

• ori exits with error 0xc0000139 on windows

  $ go run . -h
  exit status 0xc0000139
  

  This translates to STATUS_ENTRYPOINT_NOT_FOUND https://pkg.go.dev/golang.org/x/sys/windows

  This might mean you have the wrong mingw install version and windows can't find the entrypoint symbols for the libsass binary at runtime? not 100% sure but changing the mingw version to the one specified above fixes the issue.

Useful Snippets

# with mingw on your path

# Build entries
go run . -config=../build.json

# Build an entry named "OWA Mail" from the entrypoints map, with codesplitting
# Note that this has to be loaded with a script type="module" entrypoint,
# since esbuild codesplitting forces esm modules
go run . -config=../build.json -entry="OWA Mail" -split

# Generate a cpu profile for initial and incrmental builds (the traces directory must already exist)
go run . -config=../build.json -entry="OWA Mail" -traceInitialBuild -traceIncrementalBuilds -cpuprofile=traces/cpu.pprof

# Analyse cpu profiles (constains overview of CPU time)
go tool pprof -http=localhost:8080 traces/cpu.pprof.initial*
go tool pprof -http=localhost:8080 traces/cpu.pprof.incremental*

# Analyse traces
go tool trace traces/trace.out.*

FAQs

• Why not use the esbuild node API?

  In short, we tried it and it was slow. Initial build times were several minutes, compared to the 40-odd seconds we see with the go api because of all the time plugins spent waiting to run on the node main thread.

• Can I customize ori for my monorepo?

  For now, ori will remain extremly opinionared on what the monorepo shape must look like. As much as possible, we want to prefer convention over configuration.

  In the same vein, rather than implementing plugins or encouraging people to fork and make their own custom builds of ori, new functionality will be added to the same ori binaries as needed.

• Why is it called ori?

  ori was started by the Outlook Web team, and is short for OWA Rapid Innerloop.

  It can also be easily typed on a single row of a QWERTY keyboard without using your fifth fingers, which I value because I have ulnar neuropathy.

TODO: Populate this section as people ask more questions

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [e-mail] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.