Secure Website

SSL

DS Scheduler

A centralized 'cron' type scheduling system for UNIX/Linux. It has a web interface for managing, monitoring and scheduling jobs and commands in a multi-host environment. Can execute jobs based on event triggers. Secure communications between master servers and clients. Clusterable: Master can automatically fail over so any number of secondary masters. Ajax web interface for better monitoring and execution feedback. Graphing/Reporting: Visualize the running duration and timing of your jobs. DS Scheduler is released under the GPLv2.

 

Download:

Download ds-scheduler via GIT:

 GitHUB ::

 

Releases are alavilable in tar.bz2 format:

scheduler-0.7.tar.bz2

Features:

  • Can be used as a drop-in replacement for cron.
  • Can execute jobs based on event triggers.
  • Secure communications between master servers and clients.
  • Cluster-able: Master can automatically fail over to any number of secondary masters. 
  • Ajax web interface for better monitoring and execution feedback.
  • Graphing/Reporting: Visualize the running duration and timing of your jobs.
  • Run Jobs based on File Triggers
  • Database will upgrade itself.
  • Web interface will indicate if the master is running or not.

 

Installation:

Requirements:

APSchedler - available HERE

Apache Web server

Python >= 2.6

 

Point your web server to the 'web' directory that is under scheduler/.  If you installed in /opt/scheduler, then an example apache entry would be:

Alias /scheduler/ /opt/scheduler/web/
<Directory /opt/scheduler/web/>
   AuthType Basic
   AuthName "DS_SChed"
   AuthUserFile /opt/scheduler/.htpasswd
   require valid-user
</Directory>

Secure your site by using SSL and htpasswd. You can run root processes with the scheduler to restricting access is highly recommended.  

Creating an htpasswd file:

htpasswd -c /opt/scheduler/.htpasswd <user>

 

Create the Database

Create a mysql user and a blank DB called "ds-scheduler" in MySQL. Make sure to update the sched_config.py and the dbconfig.php files to reflect the database user, host, and passwd.

Note for version 0.5 and later:

The sched_master application will create and upgrade the database tables. The goal here it to avoid doing any DB maintenance to allow for very easy upgrades.

Screenshots:

Web interface:

Graphs:


Admin Console (New in 0.7):
 

Those that prefer the Command Line will be happy that I am working on a terminal based console for DS-scheduler. Just run sched_console:

Hit the H key to bring up the key commands.

Sched_admin (New in 0.7):

Managing jobs are easy to script now with this command line interface:

Usage: sched_admin [OPTIONS]

Options:
 -h , --help         Print this info
 -l, --list          List jobs in the system
 -r, --run [ id ]    Run job id immediately
 -k, --kill [ id ]   Kill running job
 -a, --add [ id ]    Add a new job
     --cron <cron notation> --name <name> --host <host> --user <user> --depends [ id ] --mode [ ON_SUCCESS | ON_FAILURE ]
 -d, --delete [ id ] Delete job completely
 -e, --enable [ id ] Enable a disabled job
 -x, --disable [ id ] Disable job from running
 -q, --quiet         Quiet mode ( good for scripting )
 

TO DO:

  • Not every feature on the web interface works yet. Anyone who knows web development could probably do better. Patches are welcome. Update: Most web features now work. Testing needed.
  • Need to add ON_NEWFILE trigger.  Done
  • Need to add ON_FILECHANGE trigger.
  • Need better reporting/graphing flexibility.
  • Need Translations

Database:

For those who want to contribute to the development, Here is an outline of the database:
 

Table:  Jobs

int(11)   AUTO_INCREMENT unique job ID
varchar(255)     Human readable job name
varchar(30)     Host that job will run on
varchar(10)     user that job will run as
datetime     The last time the job started
datetime     The last tim the job ended
int(11)     Does this job depend on another?
int(11)     See the dep_mode table
varchar(30)     'cron' type notation
int(11)     The running status
int(11)     The exit code of the job
varchar(300)     The actual command to run
tinyint(1)     Trigger the scheduler to re-read the job
text     Human readable comments
Log_Retention int     How long in days to keep run logs

Table:  dep_mode

For managing dependant modes 
 

int(11)   Unique ID
varchar(25)   Description of mode (i.e. ON_FAILURE)

Table: Results

For storing output and times for reporting
 

int(11)   Unique ID  
datetime   Time the job started  
datetime   Time the job exited  
text   Command/Job output text  
varchar(32)   Unique identifier for each job instance  

Table: Events

 

Table: db_version

This table is for keeping track of the database structure.

 

int(11)                      
int(11)                      

Comments

Hi,
I've 404 when I click on link to download DS-Scheduler.
https://solar1.net/files/scheduler-0.5.tar.bz2
This link go to this page :
http://solar1.net/files/scheduler.tar.bz2
I downloaded DS Scheduler by copying the link :
https://solar1.net/files/scheduler-0.5.tar.bz2
Thanks for all.
Bye

By Pierre-Yves (not verified)

Just checked and it should be working.

By dave

Looks cool, do i need to install anything on my servers (not the scheduler server)?

By scott (not verified)

Yes. There is an agent. It is sched_slave.py. I normally just distribute the whole thing, minus the web stuff to the servers.

By dave

awesome article

By ricardaquidley0 (not verified)

Hello,
i installed this scheduler some days ago and it's globally OK. Thanks for the job !
i have some suggestions :
- fix  "disabled" state which is not persistant after a reboot (or just a restart)
- get a "signal" when a command (disable job, start job) is submitted to the scheduler (more responsive GUI)
Have you think to open a ticketing system or put your work on GitHub ?
Excuse my poor english
 

By s0x (not verified)

Thanks s0x.
I'm working on making the GUI more responsive. Thanks for the feedback. I will fix the disabled state bug.
 

By dave

Yes it is working properly thanks for the coding part. chart library

By JacobsRobert

Thanks for this program that help a lot.
There is a bug that cannot permit to use a username longuer than 10 characters. It's is a problem because I have several users with a name longuer than that.
Do you have any solution for this ?

By Lionel (not verified)

I'll release a fix, But if you want to fix this now, you can alter your database:
 
ALTER TABLE `jobs` CHANGE `user` `user` VARCHAR( 31 ) CHARACTER SET latin1 COLLATE latin1_swedish_ci NOT NULL;

By dave

Fix is in git.

By dave

Hello,
First thanks for this soft which is really great. I installed it for making tests and I noticed some bug.
 
I cannot use an username longer than 10 caracters. When I create a job with a user wich has a name longer than that, the name is troncate... exemple : exploitation become exploitati
 
Also the remote jobs aren't executed with the user I defined... The user exist on the remote computer and when I execute the job, the sched_slave.py show the good user. But the jobs are always executed with the root user...
 
I make a test by executing this command by the user admin (echo $USER : $HOSTNAME : $HOME). And the result are always the same : root : remote-hostname : /root
 
Do you have any solution ?
 
For the record I use a Debian (Wheezy) system for the master and a Redhat (6.1) for the slave.
 

By Lionel

I tested on my system with a '/usr/bin/sleep 60' as a test user

 

testuser   14627  7980  0 15:02 pts/1    00:00:00 /usr/bin/sleep 60

 

and the slave also reports the user:

2013-10-25 15:02:29.551990 :: Received /usr/bin/sleep 60 as user testuser(5142)

The slave suprocess is getting passed the environment from your current environment, so when you "echo $USER", it's reporting the user from the current environment. See what happens when you change the command to whoami"whoami", which actually checks your UID.

Another option is to use:

"/bin/bash -l -c 'echo $USER'"

-Dave

By dave

Thanks for you answer, I totally forgot the environment variable transfert from a process to a subprocess.
After testing it appear that works properly.
 
You do not answered to my question about the limitation of the username lenght. Actually I configure a username longuer than 10 caracters is it normal ?

By Lionel

Yes, I posted a workaround for you. I also committed a fix into git. See the comments above.

By dave

Thank you.
 
I really enjoy your application and your reactivity !
 
- Lionel -

By Lionel

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.
secret-login