Code Snippets

Using Supervisord to continually execute concurrent scripts in parallel | linux

Supervisord is a great Linux program to execute scripts continually, parallelize them and keep them running 24/7 reliably.

I use this to run concurrent jobs around the clock instead of CRON. Why? Because it allows multiple processes simultaneously and, also, keeps them up and running. Configuration is simple. Follow the link below for installation instructions. I installed it with PIP ( You should then follow the instructions to create the conf file at that link.

Once installed, you can create a process in the supervisord/conf.d file (usually /etc/supervisord/conf.d).

It should run straight out of the box. But it can be confusing at first. So, my tips and an example config are below.

Sure, you can just do this with a concurrent bash script. But this method is a lot more simple and more reliable.

Example use cases:

- Load testing - e.g. run a sitemap generator file pointed at your own server with 20+ workers to see how it responds under load
- Sending emails which require parellelized PHP scripts to run fast
- As a substitute for cron when you need to keep a process running

# after installation with PIP or whichever method you choose, create the config file

# here is an example script I use to load test my website.  The script is called kitsitemap.php and generates sitemaps for all domains, and also times them to see if the server is slowing down

# this goes at the bottom of the conf.d:

command=php /_cron/phpsitemap/kitsitemap.php

# note, here I have 30 "procs", that means 30 workers, i.e. 30 concurrent executions of kitsitemap.php

# autostart is set to false otherwise the thing will run continually on boot

# once configured, you can control supervisor with commands like this:

# start supervisord
1) supervisord

# start all programs in conf.d
2) supervisordctl start all

# see status
3) supervisordctl status

# stop all processes
4) supervisordctl stop all

Published: Thu 11th January 2018
||COMMENTS|| This site proudly uses PrismJS to display code snippets

If you like this page, please share it.

Code Links - PHP, JS, CSS, Bootstrap, Bash, Emmet, IntelliJ, Sublime etc.

My personal list of links which I keep here publicly incase they help someone else one day.

Code Snippets

Snippets of PHP code I have gathered over the years. I keep them here for my own repository but also to share with others. I always link back when I remember the source.

My Github Repos

A work in progress. I am going to make as many modules that I write in my devbox public as possible.

Kit's Homepage

This site was originally intended as a test bed for code. It now includes my blog about business and powerlifting, as well as games for Spanish language learning

© 2018