Serverless Elder rendering

[jbmoelker]

So, we know Elder is designed to generate static websites 💯 . However my colleague @decrek and I think Elder could be used very well for partially static and partially dynamic websites. You've already proven Elder can be used as a server in Elderjs/template > src/server.js. So we want to take it one step further and make it serverless.

@decrek has already done some work on this. Dynamic serverless Elder rendering in a lambda function would look something like this:

const renderPage = require('../lib/render-elder-page.js');
exports.handler = async (event) => {
  const user = await getUserFromDatabase(event);
  const html = await renderPage({ 
   permalink: '/account/', 
   data: { user },
  });
  return {
    statusCode: 200,
    headers: { 'Content-Type': 'text/html' },
    body: html,
  }
}

with lib/render-elder-page.js doing the actual rendering:

const { Elder, Page } = require('@elderjs/elderjs');
const elder = new Elder({ context: 'server' });
module.exports = async function renderPage({ permalink, data = {} }) {
  await elder.bootstrap();
  const request = elder.serverLookupObject[permalink];
  const route = elder.routes[request.route];
  const hooks = [
    ...elder.hooks,
    {
      hook: 'data',
      name: 'addDynamicData',
      description: 'Adds dynamic data to data object',
      priority: 50,
      run: (opts) => ({ ...opts.data, ...data }),
    }
  ];
  const page = new Page({ ...elder, hooks, request, route });
  return await page.html();
};

The setup is working locally, but we're still having some issues on Vercel and Netlify. The hardest part in the serverless context is reaching all the relevant source and generated files. The automagic behaviour, mainly inside getConfig is making this a bit difficult. The other bit is the use of process.cwd() throughout the code base. To make Elder more flexible to use, 2 things would help a lot:

• add and use a settings.rootDir throughout the code base. So for instance const srcFolder = path.join(process.cwd(), settings.locations.srcFolder); would become const srcFolder = path.join(settings.rootDir, settings.srcDir). rootDir could still default to process.cwd(). See Proposal: simplify elder.config.js #27.
• allow passing the entire config to new Elder(config) to avoid issues caused by automagic behaviour like cosmiconfig. The automagic behavior can still be used as a default for when no config is passed.

[jbmoelker]

We could go a step further and try to pre-build and pre-bundle as much of Elder and everything needed for server-side rendering. Less moving parts during runtime is better :)

For example hooks and routes still live in the source directory (src/hooks.js and src/routes/:route/route.js). These could be bundled with Rollup as well:

simplified rollup.config.js changes:

const distDir = './___ELDER___/';

const hooksConfig = {
  input: 'src/hooks.js',
  output: {
    format: 'cjs',
    exports: 'auto',
    dir: distDir,
  },
  plugins: [
    commonjs({ sourceMap: false }),
  ],
};

const routesConfig = {
  input: ['src/routes/*/route.js'],
  output: {
    format: 'cjs',
    exports: 'auto',
    dir: path.join(distDir, 'routes/'),
  },
  plugins: [
    multiInput({ relative: 'src/routes/' }),
    commonjs({ sourceMap: false }),
  ],
};

// ...

if (production) {
  // production build does bundle splitting, minification, and babel
  configs = [...configs, ...templates, ...layouts, hooksConfig, routesConfig];

[nickreese]

@jbmoelker @decrek I dig it. We've looked at a similar solution internally 👍 before open sourcing it we did EXACTLY what you are doing with Page with lambdas.

process.cwd()

I'll be candid, just like the ./path/here/ where I include the dot. This was done everywhere because I was having issues with paths on a prior project, this pattern worked, and I just stuck with it. 100% open to the refactor stated in #27. Let's do it.

Rolling Things Up:

• Currently route.js files can have a hooks array. How do you envision this working? We can trim the API now while the project is young if the trade offs are worth it. Initially when building ElderGuide.com we had several route specific hooks, but kept forgetting about them, so we moved them all into the main hooks.js file checking the request.route to limit their execution.
• How would plugins be handled? Managing the plugin/initialization is probably the most important thing that HAS to happen at bootstrap that probably doesn't make sense to rollup. The complexity with plugins mainly relies on making sure they have their own closure. We've chosen to enforce plugin init functions to be sync so that they don't slow down bootstrap.

usePriorBootstrap

Another idea that may make the whole lambda approach better (especially with Cloudflare workers) would be to have the ability to take a prior bootstrapped state, setup the closures for the plugins, and skip all the potential DB calls required to build the allRequests array.

Over the next 3 weeks I'm going to be a bit tight on development time but I'm 100% happy to review PRs for this.

[jonatansberg]

I'm also interested in this. Especially the possibility of deploying to something like Cloudflare Workers, Lambda@edge and Akamai EdgeWorkers.

[jbmoelker]

@nickreese

Over the next 3 weeks I'm going to be a bit tight on development time but I'm 100% happy to review PRs for this.

Same here, I'm on vacation. I'll ask my @voorhoede colleagues if they can help out.

[decrek]

Hi @nickreese! ! First of all, great work on ElderJS!

I have an update on this issue. I managed to get serverless rendering working with Netlify functions and I wanted to share my example. I had to fork Elder to make some small changes in order to get it working so I want to propose some changes to Elder as well.

• Custom Elder config:
  I need a different elder config during build and during serverless rendering since paths differ. But more importantly, cosmiconfig will not find the elder config during serverless rendering since it has to look up in the folder structure and down in some other folder. The first change I made was the ability to pass the Elder config to the Elder constructor, thus bypassing consmiconfig looking up the Elder config. See commit.

Next to that, I can not rely on process.cwd() for the root of the project. In order to fix I've added a property to the elder config, which I called rootDir, based on NuxtJS rootDir property. See commit. But I looks like you already added that option?

These 2 changes give me more flexibility over configuring Elder in different environments.
As you can see these 2 changes overwrite the default behaviour. Meaning passing the config to the constructor is completely optional, falling back to cosmiconfig doing the same thing as before the change. Besides that, not setting a rootDir will result in the default value of process.cwd().

• Extending data:
  As @jbmoelker shared already with the hooks system it is perfectly possible to add data using hooks. Note, that in the simplified example I don't use this. But you can see the idea here;

• Removing SSR routes from static build:
  Netlify matches 404's to serverless function so I need to delete the statically generated HTML file. I did this using a afterbuild hook removing routes that have a ssrOnly property in the route object. This works fine, except for the fact that the file is not present in the afterBuild hook, when adding a setTimeout it worked: See here the hook.

I think the possibility to do serverless rendering using Elder is pretty awesome! What do you think about the whole approach and how do you feel about passing the configuration to the Elder constructor?

[decrek]

Oh and here is the link to the working example: https://deploy-preview-1--elderjs-serverless.netlify.app/

[nickreese]

@decrek Great work, this is exciting.

1. Definitely open to accepting a config as shown here. If you want to push a PR for just that, that would be great. 👍
2. Yep, this is exactly by design. Great usage.
3. Why not await the removal of the directory? I'd do this as defined below.

Disabling writing of only specific pages.

1. Disable the hook that writes the html (elderWriteHtmlFileToPublic) in your elder.config.js.
2. Add ssrOnly to the request objects.
3. Re-implement the elderWriteHtmlFileToPublic hook excluding any requests with ssrOnly.