Server load average
| Server | 1 min | 5 min | 15 min | Top CPU Hogs |
|---|
| bugs-vm1 | 0.32 | 0.17 | 0.15 | |
| bugs-vm2 | 0.28 | 0.20 | 0.16 | |
| db2master | 0.07 | 0.18 | 0.28 | 25.4 mysql /usr/sbin/mysqld
13.8 root /usr/bin/ruby.ruby2.1
0.1 root /usr/sbin/haveged |
| db2slave | 0.16 | 0.18 | 0.17 | 13.6 root /usr/bin/ruby.ruby2.1
11.5 mysql /usr/sbin/mysqld
0.0 root /usr/lib/systemd/systemd |
| dbapi | 0.12 | 0.24 | 0.31 | 25.4 mysql /usr/sbin/mysqld
0.0 root /usr/lib/systemd/systemd
0.0 root [kthreadd] |
| dbapislave | 0.08 | 0.13 | 0.38 | 26.7 mysql /usr/sbin/mysqld
0.0 root /usr/lib/systemd/systemd
0.0 root [kthreadd] |
| dbmaster | 2.00 | 2.01 | 2.05 | 0.3 mysql /usr/sbin/mysqld
0.0 root /usr/lib/systemd/systemd
0.0 root [kthreadd] |
| dev1 | 0.68 | 0.42 | 0.31 | 0.0 root [md0_raid1]
0.0 root /usr/sbin/irqbalance
0.0 root /usr/sbin/nscd |
| dev3 | | | | DEAD, offline or not reporting |
| download1 | 0.26 | 0.49 | 0.65 | 15.0 root /usr/bin/ruby.ruby2.1
2.4 nobody /usr/sbin/rsyncd
1.5 nobody /usr/sbin/rsyncd |
| download2 | 0.35 | 0.46 | 0.49 | 18.3 root /usr/bin/ruby.ruby2.1
1.1 wwwrun /usr/sbin/httpd-worker
1.1 wwwrun /usr/sbin/httpd-worker |
| download3 | | | | DEAD, offline or not reporting |
| econ-staging | 0.71 | 0.18 | 0.06 | 26.9 root /usr/bin/ruby.ruby2.1
0.1 mysql /usr/sbin/mysqld
0.1 root /usr/lib/systemd/systemd |
| econ-vm1 | 0.03 | 0.05 | 0.03 | 0.2 wwwrun /usr/lib/systemd/systemd
0.1 root /usr/lib/systemd/systemd
0.1 wwwrun /usr/sbin/httpd-prefork |
| econ-vm2 | 0.00 | 0.00 | 0.00 | 0.1 wwwrun /usr/lib/systemd/systemd
0.1 root [rcu_sched]
0.1 root [xfsaild/xvda1] |
| forgephp-vm1 | 1.35 | 1.16 | 1.16 | |
| forgephp-vm2 | 0.81 | 0.82 | 0.84 | |
| forgephp-vm3 | 1.23 | 1.52 | 1.54 | |
| forgephp-vm4 | 0.95 | 1.19 | 1.32 | |
| forgephp-vm5 | 1.73 | 1.28 | 1.18 | |
| forgephp-vm6 | 0.89 | 1.05 | 1.15 | |
| foundation | 0.31 | 0.28 | 0.31 | 1.2 mysql /usr/sbin/mysqld
1.0 root /sbin/bacula-fd
1.0 root [kworker/0:0] |
| fred | 0.21 | 0.57 | 0.74 | 8.5 root [nfsd]
4.1 root [nfsd]
3.5 dirsrv /usr/sbin/ns-slapd |
| gerrit-vm1 | 1.72 | 1.88 | 1.68 | 162 gerrit GerritCodeReview
38.8 root /usr/bin/ruby.ruby2.1
1.7 root [kworker/6:1H] |
| php-vm1 | 0.46 | 0.46 | 0.50 | |
| php-vm2 | 0.69 | 0.68 | 0.66 | |
| php-vm3 | 0.55 | 0.53 | 0.50 | |
| varn-vm1 | 0.46 | 0.28 | 0.21 | 9.4 solr /usr/lib64/jvm/java-1_5_0-ibm-1.5.0/jre/bin/java
0.0 root [pdflush]
0.0 ntp /usr/sbin/ntpd |
| www-staging | 0.94 | 0.86 | 0.93 | |
| www-vm1 | 1.37 | 1.30 | 1.34 | |
| www-vm2 | 2.12 | 1.62 | 1.39 | |
| www-vm3 | 1.44 | 1.45 | 1.36 | |
Server load indicates how busy our servers are at any point in time.
"main" and "main2" are backend file/database/authentication servers,
whereas node1 to node4 and utils are load-balanced web/cvs servers.
For an overall picture of system load, look at the CPU graph, above.
Server load will rarely be the cause of a performance slowdown,
but can affect certain system maintenance jobs that are
scheduled to run throughout the day.
Load average is displayed for the last minute,
the last 5 minutes and the last 15 minutes.
Which of those servers are you using? I don't know.
Your web pages, bugzilla pages and CVS connections are being
processed by any one of the "node" servers.
- 0.00 to 1.99 : no delay in processing requests. This is what you should see most of the time.
- 2.00 to 4.99: server is busy, but nothing major.
- 5.00 to 9.99: the house is rockin' - expect short processing delays.
- 10.00 and up: general server overload. It could be just a spike due to replication or disk activity, because this isn't normal.
Bandwidth usage and connection count
| Server | Mbps out | Mbps in | HTTP Conn | RSYNCd Conn | git Procs |
|---|
| bugs-vm1 | 0.05 | 0.01 | 9
| 0
| 0
|
| bugs-vm2 | 0.63 | 0.09 | 9
| 0
| 0
|
| db2master | 98.43 | 4.88 | 0
| 0
| 0
|
| db2slave | 1.77 | 0.52 | 0
| 0
| 0
|
| dbapi | 0.00 | 0.00 | 0
| 0
| 0
|
| dbapislave | 0.18 | 0.11 | 0
| 0
| 0
|
| dbmaster | 0.00 | 0.00 | 0
| 0
| 0
|
| dev1 | 0.00 | 0.00 | 31
| 0
| 0
|
| dev3 | 0 | 0 | 0 | 0 | 0 |
| download1 | 123.61 | 1.42 | 224 | 3
| 0
|
| download2 | 77.53 | 1.31 | 224 | 1
| 0
|
| download3 | 0 | 0 | 0 | 0 | 0 |
| econ-staging | 0.01 | 0.01 | 11
| 0
| 0
|
| econ-vm1 | 0.01 | 0.00 | 51
| 0
| 0
|
| econ-vm2 | 0.00 | 0.00 | 51
| 0
| 0
|
| forgephp-vm1 | 0.23 | 0.12 | 51
| 0
| 0
|
| forgephp-vm2 | 0.07 | 0.08 | 51
| 0
| 0
|
| forgephp-vm3 | 0.21 | 0.03 | 51
| 0
| 0
|
| forgephp-vm4 | 0.27 | 0.04 | 51
| 0
| 0
|
| forgephp-vm5 | 0.19 | 0.03 | 51
| 0
| 0
|
| forgephp-vm6 | 0.16 | 0.03 | 51
| 0
| 0
|
| foundation | 0.04 | 0.04 | 51
| 0
| 0
|
| gerrit-vm1 | 1.11 | 0.74 | 7
| 0
| 0
|
| php-vm1 | 0.15 | 0.02 | 51
| 0
| 0
|
| php-vm2 | 0.07 | 0.01 | 51
| 0
| 0
|
| php-vm3 | 0.11 | 0.01 | 51
| 0
| 0
|
| proxy-vm1 | 0 | 0 | 0 | 0 | 0 |
| proxy-vm2 | 0 | 0 | 0 | 0 | 0 |
| varn-vm1 | 0.00 | 0.00 | 1
| 0
| 0
|
| www-staging | 0.00 | 0.00 | 11
| 0
| 3
|
| www-vm1 | 3.19 | 0.18 | 84
| 0
| 0
|
| www-vm2 | 4.38 | 0.16 | 89
| 0
| 2
|
| www-vm3 | 2.09 | 0.12 | 82
| 0
| 2
|
| Total | 314.49 | 9.96 | 1394 | 4 | 7 |
Bandwidth usage indicates the overall usage of our Internet link.
When usage is at, or near 100%, the following services get priority access to a portion of our bandwidth, and are
generally not affected by a saturated internet connection (in no particular order): CVS, Bugzilla,
RSYNC, SSH (including SFTP and SCP), www.eclipse.org and newsgroups.
When our connection is saturated, downloading from download.eclipse.org will
be slower than CVS or our website.
MySQL server metrics
| | dbapi | dbapislave | db2master | db2slave |
|---|
| Threads | 60 | | 27 | |
| Queries_Per_Second | 102.90 | | 2059.17 | |
| Query_Cache_Hit_Rate(%) | 50.11 | | 82.53 | |
| Slave_running | OFF | | OFF | |
| Slave_IO_Running | | Yes | | Yes |
| Slave_SQL_Running | | Yes | | Yes |
| Seconds_Behind_Master |
| 0
|
| 0
|
The database servers handle all database traffic: Bugzilla, eclipse.org website, the Wiki, and others. All write operations (committing a bug, indexing the site, tracking a download) go to the master
and are replicated immediately to the slave. Some applications, such as Bugzilla, Download Stats and the Website use the slave server for
searches and lookups for increased performance.
A high load average will cause database operations to be slower. The Queries_Per_Second and Threads metrics are good indicators of
how busy the database servers are.
Legend
- Orange indicates dev/download general cluster node.
- Green indicates Bugzilla cluster node.
- Blue indicates PHP/Wiki cluster node.
- Gray indicates www.eclipse.org cluster node.
- Light red indicates Storage/database backend node.
Currently running on accounts.eclipse.org at 172.30.4.71 on Monday, June 27, 2022 - 03:45.
- CPU: 0.8%
- Bandwidth: 89.9%
- HTTPConn: 13.9%
