901-950 of 10000 results (30ms)
2014-11-11 §
14:23 <reedy> Synchronized private/PrivateSettings.php: Add $wmgVERPsecret for BounceHandler (duration: 00m 14s) [production]
14:09 <reedy> Synchronized php-1.25wmf7/extensions/BounceHandler/: (no message) (duration: 00m 15s) [production]
14:04 <reedy> Synchronized php-1.25wmf6/vendor/: (no message) (duration: 00m 15s) [production]
14:04 <reedy> Synchronized php-1.25wmf6/extensions/BounceHandler/: (no message) (duration: 00m 14s) [production]
13:41 <reedy> Synchronized wmf-config: (no message) (duration: 00m 14s) [production]
13:35 <godog> rolling reload on ms-be2* to pick up statsd changes [production]
13:12 <reedy> rebuilt wikiversions.cdb and synchronized wikiversions files: Non wikipedias to 1.25wmf7 [production]
13:11 <reedy> Purged l10n cache for 1.25wmf5 [production]
13:08 <YuviPanda> deleting tons of junk data generated by interaction between txstatsd and the labs graphite archiver on labmon1001 [production]
04:29 <LocalisationUpdate> ResourceLoader cache refresh completed at Tue Nov 11 04:29:32 UTC 2014 (duration 29m 31s) [production]
02:41 <LocalisationUpdate> completed (1.25wmf7) at 2014-11-11 02:40:58+00:00 [production]
02:28 <LocalisationUpdate> completed (1.25wmf6) at 2014-11-11 02:28:21+00:00 [production]
00:21 <catrope> Synchronized php-1.25wmf7/extensions/Flow: SWAT (duration: 00m 05s) [production]
00:21 <catrope> Synchronized php-1.25wmf7/extensions/VisualEditor: SWAT (duration: 00m 04s) [production]
2014-11-10 §
23:22 <paravoid> reprepro: include src:libmaxminddb, src:geoipupdate for precise/trusty [production]
22:14 <cscott> updated Parsoid to version b61475196 [production]
21:49 <cscott> updated OCG to version d9855961b18f550f62c0b20da70f95847a215805 [production]
21:36 <mutante> powercycling frozen stat1002 [production]
18:42 <manybubbles> restarting remaining elasticsearch boxes in sequence to pick up new plugins [production]
18:30 <godog> reboot db1017 to pick up an updated kernel [production]
18:29 <ori> Synchronized php-1.25wmf6/includes/ChangeTags.php: Iec9befeba: Hide HHVM tag on Special:{Contributions,RecentChanges,...} (duration: 00m 05s) [production]
18:29 <ori> Synchronized php-1.25wmf7/includes/ChangeTags.php: Iec9befeba: Hide HHVM tag on Special:{Contributions,RecentChanges,...} (duration: 00m 06s) [production]
17:52 <manybubbles> restart elastic1002 to pick up new plugins [production]
17:16 <manybubbles> elastic1001 finished restarting. letting is soak up shards for a few minutes to make sure restart was ok. then we'll plow through the others [production]
17:02 <demon> Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 05s) [production]
16:52 <manybubbles> restarting elastic1001 to pick up new plugins. [production]
16:50 <manybubbles> deployed new versions of elasticsearch plugins to fix regex querying [production]
16:48 <reedy> Synchronized wmf-config/InitialiseSettings.php: tocuh (duration: 00m 14s) [production]
16:03 <anomie> Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable JPG thumbnail chaining on all wikis except commons [[gerrit:172254]] (duration: 00m 09s) [production]
16:01 <anomie> Synchronized wmf-config/Wikibase.php: SWAT: Enable experimental Wikidata features on labs [[gerrit:172239]] (duration: 00m 09s) [production]
15:50 <reedy> Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 14s) [production]
11:54 <oblivian> Synchronized wmf-config/CommonSettings.php: Open HHVM to 15% of anons (duration: 00m 06s) [production]
10:06 <_joe_> upgraded hhvm on the whole cluster [production]
09:19 <hashar> Restarting Jenkins to java 7 [production]
09:17 <_joe_> upgrading hhvm across the fleet with new package with debug symbols [production]
09:14 <hashar> Jenkins: switching from Java 6 to Java 7 {{gerrit|153764}} [production]
09:02 <_joe_> repooling mw1189 at reduced load [production]
08:52 <_joe_> dist-upgrading mw1189 to use the latest kernel available, then rebooting [production]
08:32 <YuviPanda> ran mklost+found on /srv/postgres for reducing cronspam [production]
08:21 <paravoid> force-rebooting ms-be2011, kernel "xfs stuck" [production]
01:55 <ori> depooled mw1114 after it became unresponsive, likely <https://phabricator.wikimedia.org/T1195> [production]
2014-11-09 §
23:44 <hoo> Changed the email for a global account. Bug 73014. [production]
21:56 <_joe_> depooling mw1189 from the api pool, see https://phabricator.wikimedia.org/T1194 [production]
19:12 <_joe_> restarted apache on mw1192, this time an hard restart [production]
17:11 <hoo> mw1192 stuck with almost no idle workers as most workers are in the "Gracefully finishing" state. Attempted to gracefully restart it, but that (to no surprise) didn't help. [production]
2014-11-08 §
20:17 <Krinkle> Jenkins/Zuul was still stuck. Disconnected and relaunched slave agents on lanthanum and gallium. This fixed it (slaves in labs were fine). [production]
20:01 <Krinkle> Jenkins/Zuul appear stuck. Disconnect/Re-enable Gearman from Jenkins. [production]
15:30 <demon> Synchronized wmf-config/InitialiseSettings.php: Gerrit I46b151ff: Reverting addition of Draft namespace to enwiki (duration: 00m 04s) [production]
11:09 <YuviPanda> ran makelost+found on /srv/postgres on labsdb1004 to kill cronspam [production]
11:07 <YuviPanda> ran makelost+found on /srv/postgres on labsdb1007 to kill cronspam [production]