1951-2000 of 10000 results (31ms)
2015-09-24 §
02:14 <Krinkle> Partial EventLogging outage (client-side events via hafnium abruptly stopped 2015-09-23 11:36 UTC - 15 hours ago) [production]
01:53 <mutante> started logstash on logstash1002 again [production]
01:35 <mutante> bast1001: unmounting /srv/home_pmtpa (backup on bacula) [production]
01:34 <mutante> removing subversion packages from bast1001 [production]
01:15 <ori@tin> Synchronized php-1.26wmf24/includes: Ifa0d4cfe8e3: Backport I1ff61153d and I8e4c3d5a5 (duration: 00m 23s) [production]
00:19 <jynus> restarted replication on db1051 [production]
00:17 <ori> restarting tcpircbot on neon [production]
00:16 <mutante> started logstash on logstash1002 [production]
00:08 <bblack> varnish package on carbon for jessie updated to 3.0.6plus-wm8 [production]
2015-09-23 §
23:17 <krenair@tin> Synchronized php-1.26wmf24/includes/specials/SpecialSearch.php: https://gerrit.wikimedia.org/r/#/c/240596/ (duration: 00m 18s) [production]
23:09 <krenair@tin> Synchronized wmf-config/throttle.php: https://gerrit.wikimedia.org/r/#/c/240575/ (duration: 00m 17s) [production]
22:53 <ori@tin> Synchronized php-1.26wmf24/includes/resourceloader/ResourceLoaderFileModule.php: 14f46330d9: Backport fix from PS12 of I1ff6115 (duration: 00m 17s) [production]
22:39 <paravoid> cr1-codfw RE switchover(s) [production]
22:35 <yurik@tin> Synchronized php-1.26wmf24/extensions/ZeroBanner: Deploying ZeroBanner interstitial handling (duration: 00m 18s) [production]
21:13 <ori@tin> Synchronized php-1.26wmf24/includes/resourceloader/ResourceLoaderFileModule.php: 58bfb6f85b: Backport fix from PS9 of I1ff6115 (duration: 00m 17s) [production]
20:46 <subbu> deployed parsoid 6619409e [production]
19:58 <urandom> starting Cassandra on restbase2006 [production]
19:56 <urandom> enabling puppet, and forcing a run on restbase2006 [production]
19:52 <urandom> starting Cassandra on restbase2005 [production]
19:50 <urandom> enabling puppet, and forcing a run on restbase2005 [production]
19:48 <urandom> starting Cassandra on restbase2004 [production]
19:45 <urandom> enabling puppet, and forcing a run on restbase2004 [production]
18:55 <twentyafterfour> snapshot1001.eqiad.wmnet returned [12]: rsync: write failed on "/srv/mediawiki/wikiversions.cdb": No space left on device [production]
18:54 <twentyafterfour@tin> rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.26wmf24 [production]
18:01 <mobrovac> restbase deploying f65313ed [production]
17:49 <urandom> starting Cassandra on restbase2003 [production]
17:45 <demon@tin> Synchronized php-1.26wmf23/extensions/Wikidata: (no message) (duration: 00m 25s) [production]
17:35 <urandom> enabling, and forcing puppet run on restbase2003 [production]
17:27 <demon@tin> Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 13s) [production]
17:20 <ori@tin> scap failed: OSError [Errno 2] No such file or directory: '/var/lock/scap' (duration: 44m 24s) [production]
17:11 <ori@tin> Synchronized php-1.26wmf24/includes/page/Article.php: (no message) (duration: 00m 13s) [production]
17:11 <ori@tin> Synchronized php-1.26wmf23/includes/page/Article.php: (no message) (duration: 00m 17s) [production]
16:35 <ori@tin> Started scap: (no message) [production]
16:35 <ori@tin> Synchronized php-1.26wmf23/includes/page: I952068d2d: Reduced the DOS potential of 404 page floods (duration: 00m 12s) [production]
16:22 <godog> start cassandra on restbase2002 [production]
16:04 <godog> deploy restbase daacf4d on restbase2* [production]
15:34 <demon@tin> Synchronized php-1.26wmf24/extensions/Wikidata: (no message) (duration: 00m 20s) [production]
15:34 <demon@tin> Synchronized php-1.26wmf23/extensions/Wikidata: (no message) (duration: 00m 21s) [production]
15:34 <chasemp> unbanning elastic1005 for T112559 [production]
15:32 <demon@tin> scap aborted: (no message) (duration: 02m 07s) [production]
15:30 <demon@tin> Started scap: (no message) [production]
15:15 <demon@tin> Synchronized php-1.26wmf22/extensions/Wikidata: (no message) (duration: 00m 20s) [production]
14:13 <demon@tin> Synchronized wmf-config/abusefilter.php: (no message) (duration: 00m 12s) [production]
14:08 <demon@tin> Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 12s) [production]
14:05 <demon@tin> Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 12s) [production]
14:04 <demon@tin> Synchronized langlist-labs: (no message) (duration: 00m 11s) [production]
14:04 <demon@tin> Synchronized docroot/noc/conf/: (no message) (duration: 00m 13s) [production]
14:01 <demon@tin> Synchronized multiversion/MWRealm.php: (no message) (duration: 00m 11s) [production]
13:56 <godog> force puppet run on restbase2001 to deploy new cassandra config [production]
13:28 <godog> reboot ms-be1012, xfs hosed [production]