Dependent services and sequential deploys | Feature Requests

Dependent services and sequential deploys

Dan Wendorf

Sometimes, a service is dependent on another service, like when a background worker is dependent on the main service getting updated and running migrations. It would be nice to be able to define this relationship in Render, so things like deploys can run sequentially.

April 12, 2021

kate

marked this post as

open

Peter Nixey

Just adding my 2c to this. I've got a v similar scenario as this (rails back end being built and an angular front end - which I'd like to deploy simultaneously). 
The thing that I'd love is the ability to trigger a build separately from switching the traffic to the new build. 
As far as I understand it, when Render deploys, it builds the updated service and then once it's fully built, it switches the traffic from the old service to the new service and before finally shutting down the old one. 
I'd love to have API-access to those final steps so that I could allow all services to be built, confirm that's happened successfully and then switch traffic over to all services simultaneously.

Sven Schwyn

Here's how we work around this for the time being: A simple CLI tool "build-checkpoint" waits until all builds (say web, assets etc) have reached the checkpoint in their build scripts. When this is the case (means: when the checkpoint has been hit as many times as the NUMBER you passed it), all builds scripts continue and therefore deploy roughly at the same time. And if one build fails, the all fail. For this to work, you have to set BUILD_RENDER_URL to a Render service (internal URL is okay). Furthermore, the services are grouped by their RAILS_ENV. You'd have to tweak this if you're using another framework. Just comment on the gist if you have a question.

https://gist.github.com/svoop/8e09c82d79169cff73ba071ab2adaf80

Mike Wille

This is a clever solution. 
However, for anyone implementing this, there is a caveat to be aware of. If you modify any setting in Render on any single service (say a health check setting or an environment variable), then that will trigger a new build/deployment for that service. At which point the build checkpoint script runs. But because there are no other services being deployed, it will wait in vain until the timeout hits. At which point, the build will fail.
Any setting change will cause a re-deploy and you can't change multiple settings at once. This situation is a mess.

Sven Schwyn

Mike Wille Yes, that's absolutely true and the reason why I'm so desperately waiting for a proper implementation by Render. My gist is a "better than nothing" workaround, nothing more. However, here's a trick to prevent the mess: When for whatever reason a single service rebuilds, you have to trigger a rebuild of all related services ASAP. The easiest way to do this is to define an environment group and add all related services to this group. In the environment group, add an environment variable like

DEPLOY_NONCE

and assign it any value. To trigger a rebuild of all services of the group, just change the value of

DEPLOY_NONCE

and Render will trigger the rebuild on all services in the environment group. Hacky, sure, but hopefully we don't need these workarounds much longer.

Sven Schwyn

Since this is planned, it would be very helpful to get an update on how far up it is on the TODO list. We need a solution for this soonish and implementing it ourselves into the build scripts would not only be flaky, but also superfluous once this feature has been rolled out. However, we can't migrate a few remaining things to Render unless there's a solution to this, so we'd really need a word in this matter.

Carolin Maisenbacher

We are having a similar issue and I think dependent services could be the answer.

We have multiple projects with a Monorepo setup that includes:

Node.js backend
Postgres db instance
Next.js Frontend Application

Currently the build of the build of the Frontend is much faster than the Backend build. Which leads to errors, because, for a few minutes, it causes a miss-match of the api and the dashboard logic.

It would be very useful if we could wait for both builds to finish, so the applications can be deployed at the same time.

Matthew Conto

An implementation of this could also respond to the need for a Release phase script

Felix Christl

This would be awesome to have. We have a headless CMS (Strapi) and a site generator (Next.js). Next.js can only build when Strapi has finished deploying. At the moment, Next.js builds faster than Strapi and every Next.js build fails as Strapi is not yet running. We solved it by disabling auto deploy for Strapi for now, but a dependency would be much nicer.

Marc Köhlbrugge

I've thought about this a bit more and I'm not sure sequential deploys are the best solution for the database migration scenario.
I wouldn't want my background worker to (temporarily) run on older codebase than my web service. That could lead to all kinds of problems.
Ideally the web service and worker get deployed simultaneously (and there should be no scenarios where one succeeds and the other fails). But there still would be a way to run a script (such as a database migration) that runs once after both are deployed.
It seems to me Render would really benefit from the concept of a 'project' that could be collections of multiple services, databases, etc. With its own hooks like a post-project-deploy script. This would also make other feature requests easier to solve such as project-based metrics.

Štefan Ľupták

It's probably not exactly this case, but maybe it could be considered together with this feature request.

It would be so useful for us to be able to specify service defined in another one of our repos as dependent.

For example, we have:

frontend defined in render.yaml in repository A
backend defined in render.yaml in repository B

I would like to have separate preview environment containing frontend and backend when a PR is created on one of those repositories without the need to duplicate the whole definition of a service in both render.yaml files. 🙏

Marc Köhlbrugge

+1 GitHub Actions has something like this as well: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idneeds

→