5901-5950 of 10000 results (32ms)
2015-05-03 §
12:47 <_joe_> restarting redis server on rdb1001, lagging on the most basic queries [production]
12:38 <_joe_> deploying I969fe8d329c1bcbb919a54cb225200ba0e006a03 to the jobrunners trying to make them work again [production]
05:14 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun May 3 05:13:13 UTC 2015 (duration 13m 12s) [production]
04:28 <springle> xtrabackup clone db1049 to db1070 [production]
04:01 <springle> Synchronized wmf-config/db-eqiad.php: depool db1070 (duration: 00m 16s) [production]
02:48 <LocalisationUpdate> completed (1.26wmf4) at 2015-05-03 02:47:30+00:00 [production]
02:47 <springle> Synchronized wmf-config/db-eqiad.php: repool db1068, warm up (duration: 00m 15s) [production]
02:44 <l10nupdate> Synchronized php-1.26wmf4/cache/l10n: (no message) (duration: 07m 11s) [production]
02:27 <LocalisationUpdate> completed (1.26wmf3) at 2015-05-03 02:26:02+00:00 [production]
02:22 <l10nupdate> Synchronized php-1.26wmf3/cache/l10n: (no message) (duration: 08m 11s) [production]
2015-05-02 §
22:16 <ori> Deployed change I3bc87f3a5 to fix UBN! bug T97912. Bug was affecting ability to translate messages needed for running upcoming board election. [production]
22:16 <ori> Synchronized php-1.26wmf4/extensions/Translate/api/ApiQueryMessageGroups.php: I3bc87f3a5: ApiQueryMessageGroups: mark '_canchange' and '_name' as non-API-metadata (duration: 00m 30s) [production]
22:09 <ori> Synchronized php-1.26wmf3/extensions/Translate/api/ApiQueryMessageGroups.php: I3bc87f3a5: ApiQueryMessageGroups: mark '_canchange' and '_name' as non-API-metadata (duration: 00m 31s) [production]
20:25 <windowcat> Updated jobrunners to c95d565e242e6fa3706c088ddab1cc6f716408e1 [production]
19:31 <springle> xtrabackup clone db2048, db2049, db2050, db2051, db2052, db2053, db2054 from codfw masters [production]
19:10 <springle> upgrade db1068 trusty, xtrabackup clone from db1056 [production]
19:03 <ottomata> resinstalling analytics1004 and analytics1010 as trusty [production]
06:08 <yuvipanda> signed puppet certs manually on virt1000 [production]
05:19 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat May 2 05:18:29 UTC 2015 (duration 18m 28s) [production]
03:24 <ori> Granted self admin rights on metawiki temporarily to debug a CentralNotice issue. [production]
02:53 <LocalisationUpdate> completed (1.26wmf4) at 2015-05-02 02:52:36+00:00 [production]
02:48 <l10nupdate> Synchronized php-1.26wmf4/cache/l10n: (no message) (duration: 07m 01s) [production]
02:32 <catrope> Synchronized php-1.26wmf3/extensions/WikiEditor: Fix data gathering bug (duration: 00m 25s) [production]
02:32 <LocalisationUpdate> completed (1.26wmf3) at 2015-05-02 02:31:00+00:00 [production]
02:27 <l10nupdate> Synchronized php-1.26wmf3/cache/l10n: (no message) (duration: 08m 11s) [production]
02:16 <catrope> Synchronized php-1.26wmf4/extensions/WikiEditor: Fix data gathering bug (duration: 00m 15s) [production]
00:02 <ori> Synchronized wmf-config/CommonSettings.php: (no message) (duration: 00m 16s) [production]
2015-05-01 §
23:53 <aaron> Synchronized php-1.26wmf4/includes/media/DjVu.php: caa2efc0e76c2ba849d465006600d131dc2f78b5 (duration: 00m 21s) [production]
23:52 <aaron> Synchronized php-1.26wmf3/includes/media/DjVu.php: 6cdb23c5d662151a2b578c2acc8823bc975fc22a (duration: 00m 15s) [production]
23:40 <ori> Synchronized wmf-config/InitialiseSettings.php: I02e28db61: Update apple-touch to use static (duration: 00m 23s) [production]
21:08 <matt_flaschen> Ran FlowUpdateWorkflowPageId.php for all production Flow wikis for https://phabricator.wikimedia.org/T96888 [production]
20:37 <andyrussg> Synchronized php-1.26wmf4/extensions/EducationProgram/: Update EducationProgram (duration: 00m 21s) [production]
20:01 <demon> Synchronized wmf-config/CommonSettings.php: less realm stuff (duration: 00m 17s) [production]
20:00 <andyrussg> Synchronized php-1.26wmf3/extensions/EducationProgram/: Update EducatiDonProgram (duration: 00m 30s) [production]
18:54 <legoktm> Synchronized wikiversions-labs.json: https://gerrit.wikimedia.org/r/#/c/208170/ no-op (duration: 00m 25s) [production]
18:53 <legoktm> Synchronized all-labs.dblist: https://gerrit.wikimedia.org/r/#/c/208170/ no-op (duration: 00m 18s) [production]
18:11 <legoktm> Synchronized all-labs.dblist: https://gerrit.wikimedia.org/r/208154 - no-op (duration: 00m 19s) [production]
15:58 <anomie> Synchronized php-1.26wmf3/includes/: Deploy [[gerrit:208109]] to reduce the complaining about the new feature (duration: 00m 28s) [production]
15:50 <anomie> Synchronized php-1.26wmf4/includes/: Deploy [[gerrit:208109]] to reduce the complaining about the new feature (duration: 00m 24s) [production]
15:29 <gwicke> finished restarting cassandra nodes on restbase100*.eqiad [production]
15:21 <ottomata> doing java security update on kafka brokers, doing rolling restarts [production]
14:50 <gwicke> slowly restarting restbase100*.eqiad to apply new gen size change [production]
10:47 <godog> bounce apache2 on strontium [production]
10:47 <godog> bounce apache2 on palladium, mod_passenger died [production]
05:45 <LocalisationUpdate> ResourceLoader cache refresh completed at Fri May 1 05:44:23 UTC 2015 (duration 44m 22s) [production]
03:05 <LocalisationUpdate> completed (1.26wmf4) at 2015-05-01 03:04:21+00:00 [production]
03:01 <l10nupdate> Synchronized php-1.26wmf4/cache/l10n: (no message) (duration: 06m 45s) [production]
02:38 <LocalisationUpdate> completed (1.26wmf3) at 2015-05-01 02:37:20+00:00 [production]
02:31 <l10nupdate> Synchronized php-1.26wmf3/cache/l10n: (no message) (duration: 09m 46s) [production]
00:18 <catrope> Synchronized php-1.26wmf3/extensions/PageTriage/: SWAT (duration: 00m 30s) [production]