1901-1950 of 10000 results (32ms)
2015-06-08 §
14:24 <jynus> Synchronized wmf-config/db-eqiad.php: depool es1005 (duration: 01m 08s) [production]
14:23 <Coren> rsync in progress between labstore1001:store/backup and labstore1002:backup/backup (at ionice idle) [production]
14:13 <Coren> created store/backup snapshot on labstore1001 for backup copy [production]
13:03 <moritzm> added strongswan_5.3.0-1+wmf2 to jessie-wikimedia on carbon [production]
11:42 <_joe_> purging squid cache on carbon [production]
11:26 <moritzm> updated mc2* to 2:2.8.17-1+deb8u1 [production]
10:55 <jynus> Synchronized wmf-config/db-eqiad.php: repool es1007 (duration: 01m 08s) [production]
10:27 <akosiaris> disabled puppet on uranium, investigating ganglia problems [production]
10:05 <akosiaris> ganglia gmetad problems [production]
05:25 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Jun 8 05:24:08 UTC 2015 (duration 24m 7s) [production]
02:26 <LocalisationUpdate> completed (1.26wmf8) at 2015-06-08 02:25:12+00:00 [production]
02:21 <l10nupdate> Synchronized php-1.26wmf8/cache/l10n: (no message) (duration: 07m 07s) [production]
2015-06-07 §
23:27 <godog> reboot ms-be2008 sdg failed, xfs unhappy [production]
07:03 <springle> Synchronized wmf-config/db-eqiad.php: repool db1073, warm up (duration: 01m 09s) [production]
05:16 <andrewbogott> we did a whole lot of things to labstore1001 while morebots was away [production]
05:14 <andrewbogott> service nfs-kernel-server restart on labstore1001 [production]
02:26 <LocalisationUpdate> completed (1.26wmf8) at 2015-06-07 02:25:13+00:00 [production]
02:21 <l10nupdate> Synchronized php-1.26wmf8/cache/l10n: (no message) (duration: 07m 09s) [production]
2015-06-06 §
23:46 <subbu> deployed parsoid 5172a446 (cherry-pick of 719c736f) -- hotfix for T101599 [production]
05:48 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Jun 6 05:47:40 UTC 2015 (duration 47m 39s) [production]
02:31 <LocalisationUpdate> completed (1.26wmf8) at 2015-06-06 02:30:24+00:00 [production]
02:26 <l10nupdate> Synchronized php-1.26wmf8/cache/l10n: (no message) (duration: 07m 10s) [production]
2015-06-05 §
22:42 <godog> powercycle graphite2001, no console no ssh [production]
22:06 <andrewbogott> restarted apache on virt1000 [production]
20:49 <ori> Upgrading hhvm-fss on application servers to 1.1.7; expect brief 5xx spike. [production]
20:14 <demon> Synchronized php-1.26wmf8: live hack (duration: 02m 32s) [production]
20:10 <mutante> apt-get upgrade on terbium [production]
19:52 <godog> bounce redis on rdb1001/rdb1003 to pick up new slave limits [production]
19:51 <mutante> chown root:root / on terbium [production]
19:50 <godog> bounce redis on rdb1002/rdb1004 to pick up new slave limits [production]
19:29 <godog> bounce redis again on rdb1003 after increasing the slave limits more [production]
19:17 <godog> bounce redis on rdb1003 after bumping slave limits [production]
19:07 <godog> redis master logs shows periodic 'cmd=sync scheduled to be closed ASAP for overcoming of output buffer limits.' indicating the slave fails to sync [production]
18:40 <godog> spike in redis network starting at ~15.00 UTC, correlates with ocg failures [production]
18:01 <moritzm> restarted gerrit on ytterbium for java update [production]
14:43 <jynus> short lag period on db1049, traffic automatically redirected to other slave and back to normal [production]
14:07 <moritzm> added ubuntu-meta-1.325+wmf1 for trusty-wikimedia to apt.wikimedia.org (T100004) [production]
14:07 <moritzm> added ubuntu-meta-1.267.1+wmf1 for precise-wikimedia to apt.wikimedia.org (T100004) [production]
12:44 <jynus> Synchronized wmf-config/db-eqiad.php: Depool es1007 (duration: 01m 08s) [production]
12:08 <jynus> Synchronized wmf-config/db-eqiad.php: Repool es1009 (duration: 01m 08s) [production]
11:30 <_joe_> uploaded new HHVM package, installing on mw1025 for testing [production]
09:17 <moritzm> added redis_2.6.13-1+wmf1 to precise-wikimedia on apt.wikimedia.org [production]
06:24 <moritzm> added redis_2.8.4-2+wmf1 to trusty-wikimedia on apt.wikimedia.org [production]
05:24 <LocalisationUpdate> ResourceLoader cache refresh completed at Fri Jun 5 05:22:50 UTC 2015 (duration 22m 49s) [production]
04:10 <springle> Synchronized wmf-config/db-eqiad.php: depool db1073 (duration: 01m 08s) [production]
02:26 <LocalisationUpdate> completed (1.26wmf8) at 2015-06-05 02:25:20+00:00 [production]
02:21 <l10nupdate> Synchronized php-1.26wmf8/cache/l10n: (no message) (duration: 07m 09s) [production]
01:27 <tgr> deploying schema changes for Gather on enwiki, enwikivoyage, hewiki (T98490, T101460) [production]
00:08 <catrope> Synchronized php-1.26wmf8/vendor/oojs/oojs-ui/php/Tag.php: Fix OOUI fatals (T99210) (duration: 00m 13s) [production]
2015-06-04 §
23:40 <catrope> Synchronized php-1.26wmf8/extensions/MobileFrontend: SWAT (duration: 00m 13s) [production]