tom_hartley0 views
A script runs every 15 minutes from cron. Last week one run took 25 minutes, the next one started while it was still going, and both wrote to the same file.
How do I stop a second copy from starting while the first is still running?
Be kind and specific. Read the community guidelines.
Accepted answer
Use a lock around the command. flock takes a lock on a file and, with -n, gives up immediately if somebody else holds it:
*/15 * * * * flock -n /tmp/nightly.lock /usr/bin/python3 /opt/jobs/load.py >> /var/log/load.log 2>&1
If the previous run is still going, the new one exits at once and tries again 15 minutes later. The lock is released by the system when the process ends, even if it crashes, so a stale lock file cannot block you for ever.
Two more habits help: write the output to a temporary file and rename it when finished, so a reader never sees a half-written file, and make the job safe to run twice (see the thread about doubled rows).