lander

Lander

.. image:: https://img.shields.io/pypi/v/lander.svg :target: https://pypi.python.org/pypi/lander :alt: Python Package Index .. image:: https://img.shields.io/travis/lsst-sqre/lander.svg :target: https://travis-ci.org/lsst-sqre/lander :alt: Travis CI build status

HTML landing page generator for LSST PDF documentation deployed from Git to LSST the Docs.

Installation

Lander works with Python 3.5 or above. You can install it from PyPI::

pip install lander

Run lander -h for command line help.

Usage

Basic usage

To create a landing page website, run lander with the local PDF file's path::

lander --pdf

The built PDF landing page site is available, by default, from the _build directory. View the site in a browser by running a Python web server::

cd _build && python -m http.server 8000 --bind 127.0.0.1

Get metadata from an lsstdoc document

With the --lsstdoc <tex path> argument, Lander will attempt to scrape metadata from the source of a lsstdoc-class LaTeX file, including:

• abstract
• authors
• document handle
• title

See https://lsst-texmf.lsst.io for information about the lsstdoc class.

Get metadata from the Travis environment

If you're running on Travis CI, set the --env=travis to get metadata from Travis's environment variables:

• $TRAVIS_COMMIT
• $TRAVIS_BRANCH
• $TRAVIS_TAG
• $TRAVIS_REPO_SLUG
• $TRAVIS_JOB_NUMBER

Overriding metadata

Lander tries to get as much metadata from the environment as possible (including from the Git repository and the LaTeX document itself), but sometimes this isn't possible. In this case you can explicitly set metadata with these flags (see lander -h for more information):

• --abstract
• --authors (see note)
• --title
• --handle (such as LDM-151)
• --repo-url (such as https://github.com/lsst/ldm-151)
• --repo-branch (such as master)
• --date (such as 2017-05-22)
• --docushare-url (prefer the multi-version form, https://ls.st/ldm-151*)

--authors should be a JSON-formatted array, even for a single author. For example::

--authors "["First Author", "Second Author"]"

Distributing extra files from the landing page

To include ancillary files with the main PDF document, provide their file paths with the --extra-download argument. These extra files are listed in the Downloads section of the landing page. The main PDF is always included first in this list.

For example::

--extra-download demo.ipynb

Uploading to LSST the Docs

Lander works well with LSST the Docs. Lander can upload pages directly to LSST the Docs for you with these configurations:

• --upload — provide this flag to indicate a build should be uploaded.
• --ltd-product — Name of the product on LSST the Docs.
• --keeper-url or $LTD_KEEPER_URL.
• --keeper-user or $LTD_KEEPER_USER.
• --keeper-password or $LTD_KEEPER_PASSWORD.
• --aws-id or $LTD_AWS_ID.
• --aws-secret or $LTD_AWS_SECRET.

Note: these are advanced configurations and are typically added to a CI configuration automatically or by a Documentation Engineer. Reach out to #dm-docs <https://lsstc.slack.com/messages/C2B6DQBAL/>_ on Slack for help.

Development workflow

You need both Python 3.5+ and node.js_ to develop Lander.

Initial set up

Clone and install dependencies (use a Python virtual environment of your choice)::

git clone https://github.com/lsst-sqre/lander cd lander npm install -g gulp-cli npm install gulp assets pip install -e ".[dev]"

Run Python tests and linting

We use pytest for unit testing and style checks::

make pytest

You can also run end-to-end trials of landing page builds::

make test

These integration tests clone real LSST documents, compiles them with Docker, and builds landing pages to simulate continuous delivery workflows in production. Look for sites in _tests.

Build a development site

The default gulp_ workflow creates website assets and generates a test website::

gulp

This gulp task runs a browsersync_ server and refreshes the page whenever CSS, JavaScript, or HTML assets change.

Only build assets

If you want to only build CSS, icon, and JavaScript assets, run this task::

gulp assets --env=deploy

This is how assets are built on CI for releases of Lander.

Developing CSS/Sass with squared

Lander uses squared_ for visual design. All Lander CSS should be committed to the squared_ repo so that LSST SQuaRE web projects share a common visual language.

To make it easier to write Sass in squared_ while developing landing pages in Lander, we recommend linking a clone of squared_ to Lander's node_modules. Assuming you're starting from the lander/ root directory::

git clone https://github.com/lsst-sqre/squared ../squared npm link ../squared

Some patterns:

• If you're working on a branch in squared_, then update squared's version in package.json to that branch. For example: "squared": "lsst-sqre/squared#tickets/DM-10503". This allows Travis to install the development version of squared_ when testing Lander. Remember to make a release of squared_ before releasing a new version of Lander, see below.

• scss/app.scss in the lander repo imports Sass partials from squared_ and other packages (including inuitcss_).

Release workflow

1. If squared_ was modified, create a squared_ release first.
2. Update package.json with the released version of squared_. Using tagged npm releases is preferred to GitHub branches to make builds of releases repeatable.
3. Create a signed tag: git tag -s 0.1.0 -m "v0.1.0". Use the PEP 440_ schema.
4. Push the tag: git push --tags. This will automatically create a Lander release on PyPI.
5. Merge the development branch as necessary.

License

This project is open sourced under the MIT license. See LICENSE <./LICENSE>_ for details.

.. _squared: https://github.com/lsst-sqre/squared .. _PEP 440: https://www.python.org/dev/peps/pep-0440/ .. _inuitcss: https://github.com/inuitcss/inuitcss .. _browsersync: https://browsersync.io .. _gulp: http://gulpjs.com .. _node.js: https://nodejs.org/en/

########## Change Log ##########

1.0.6 (2023-07-04)

• Pin pydantic<2.0.0.
• Drop support for Python 3.7.

1.0.1 (2021-05-11)

• Pin click<8 since Click 8.0.0 currently triggers the --version mode, regardless of whether it was set on the command-line or not.

1.0.0 (2021-01-06)

This is the first release of Lander! The version 1 release line is based on the 0.1 releases, and this line will continue to be maintained for existing users of Lander.

A new version of Lander is under development, version 2.0, that will include a plugin interface for adding custom document parsers and website themes.

0.1.16 (2019-07-04)

• Update Jinja to >=2.10.1 to address CVE-2019-10906.
• Update asset build pipeline to Gulp 4 and update all other npm dependencies at the same time.

0.1.15 (2019-04-02)

• Update to lsst-projectmeta-kit to 0.3.5 for better author parsing.
• Update requests to 2.20.0 (security).

0.1.14 (2019-02-18)

• Lander will now cleanly abort when building on Travis CI, but the secure environment variables are not available. This happens if the build is triggered by a fork.
• Switched to use $TRAVIS_BUILD_WEB_URL to get a URL to the build. This is better than computing the build URL during the travis-ci.org to travis-ci.com transition.
• Updated test dependencies to pytest 4.2.1.
• Switched to setuptools_scm for version string management.

0.1.13 (2018-11-26)

• Update to lsst-projectmeta-kit 0.3.3 for improved detection of \input and \include commands in TeX source.

0.1.12 (2018-02-10)

• Update to lsst-projectmeta-kit 0.3.1 for more reliable metadata.jsonld generation (works around pandoc issues converting some documents to plain text). There's a new integration test make dmtn070 that demos this.
• Improve the testing strategy:
  • Run make pytest to run pytest with the correct arguments instead of using --add-opts in setup.cfg. This lets us run pytest directly with ad hoc arguments.
  • Run make test to run both pytest and the integration tests.

0.1.11 (2018-02-07)

• Lander now creates and uploads a metadata.jsonld document alongside the landing page content (e.g., index.html). This content can be ingested by other tools into the LSST Projectmeta database.
• Switch from metasrc to lsst-projectmeta-kit 0.3.0. It's the same package, but the new and rebranded lsst-projectmeta-kit includes the ability to generate JSON-LD from an LsstLatexDoc object.

0.1.10 (2017-12-11)

• Change known domain for DocuShare from docushare.lsstcorp.org to docushare.lsst.org. This ensures that ls.st short links can continue to be verified.
• Create default values for abstract_plain and title_plain. This fixes cases where the abstract is not set in the underlying LaTeX document.

0.1.9 (2017-11-20)

• Update metasrc to 0.2.2 to resolve issues with auto-downloading Pandoc in Travis CI (DM-12569 <https://jira.lsstcorp.org/browse/DM-12569>_).
• Update pytest to 3.2.5 and pytest-flake8 to 0.9.1 to fix incompatibilities in the floating indirect dependencies.

0.1.8 (2017-10-09)

• Update metasrc to 0.2.1
• Use metasrc's LsstLatexDoc.revision_datetime to obtain the date of a document. This method uses a combination of parsing the \date LaTeX command, looking at content Git commits, and falling back to 'now' to get an appropriate timestamp.

0.1.7 (2017-09-28)

• Update metasrc to 0.2.0. This provides Pandoc integration for improved HTML rendering of content extracted from LaTeX documents.
• Improve how loggers are configured (warning level for third-party loggers, and info/debug levels for first-party LSST SQuaRE code).

0.1.6 (2017-09-07)

• Update metasrc to 0.1.4. This update provides improved LaTeX command metadata extraction. (DM-11821 <https://jira.lsstcorp.org/browse/DM-11821>_)
• Temporarily skip ls.st and DocuShare-related unit tests because ls.st links to DocuShare are broken due to the DocuShare upgrade.

0.1.5 (2017-07-12)

• Pin to metasrc 0.1.3
• Via metasrc, Lander has improved LaTeX source processing, including handling of referenced source files (\input and \include) and macros (\def and \newcommand).
• Improved treatment of draft status. The heuristic is that a document is considered a draft if the branch is not master and lsstdraft is not present in a lsstdoc document's options.

0.1.4 (2017-07-06)

• Fix logic for determining it Lander is running in a Travis PR environment.
• Log the Lander version at startup.

0.1.3 (2017-07-02)

• Fixed Travis deployment issue. Used skip_cleanup: true to .travis.yml to prevent CSS and JS assets from bring cleaned up before creating a release.

0.1.2 (2017-06-27)

• Detect if running from a Travis PR build (using the TRAVIS_PULL_REQUEST environment variable) and if so, abort the page build and upload. This is to prevent duplicate uploads from both branch and PR-based Travis jobs.
• Pin inuitcss to 6.0.0-beta4 because of the removal of rem functions in beta5.

0.1.1 (2017-06-17)

• Update to metasrc>=0.1.1,<0.2.
• Use remove_comments and remove_trailing_whitespace feature from metasrc. This improves the accuracy of metadata extraction from tex source. For example, comment characters won't appear in extract abstract content.

0.1.0 (2017-05-24)

Initial version.

• Native PDF display via PDFObject <https://pdfobject.com>_.
• Multi-level metadata model for populating the landing page. Lander uses https://github.com/lsst-sqre/metasrc to extract content from the tex source (more work on this is needed). Otherwise, it gets data from environment variables (including Travis CI variables). Finally, metadata can be specified explicitly with command line arguments.
• Uses https://github.com/lsst-sqre/squared for CSS, icons, and logos. A Gulp and webpack workflow build these sources into deployable dependencies. Gulp and webpack are only needed by developers and CI; the deployable artifacts are included in PyPI releases.
• Release workflow is fully automated in Travis. Create a PEP 440 version tag, push it, and the release appears in PyPI.
• Includes an LSST the Docs upload client (via https://github.com/lsst-sqre/ltd-conveyor) built-in so that https://github.com/lsst-sqre/ltd-mason isn't required.
• Usage and development docs are currently in the README.

MIT License

Copyright (c) 2017 Association of Universities for Research in Astronomy, Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.