|
2015-05-19
§
|
| 17:44 |
<robh> |
mailing lists still down, scrubbing list archives is painful and error prone |
[production] |
| 17:34 |
<ottomata> |
starting reboots of analytics worker nodes in order to enable hyperthreading Bug: https://phabricator.wikimedia.org/T90640 |
[production] |
| 17:06 |
<robh> |
puppet stopped on sodium (dont need it restarting mailman while im working) |
[production] |
| 17:05 |
<robh> |
starting mailman downtime window to scrub content off list archive per T99098 |
[production] |
| 17:00 |
<bblack> |
automated reboots of esams/eqiad non-upload caches starting (should auto-downtime, should be no real impact)... |
[production] |
| 15:51 |
<anomie> |
Synchronized php-1.26wmf5/extensions/AbuseFilter/: SWAT: Fix boolean response in API action=abusefiltercheckmatch [[gerrit:211743]] (duration: 00m 12s) |
[production] |
| 15:50 |
<anomie> |
Synchronized php-1.26wmf6/extensions/AbuseFilter/: SWAT: Fix boolean response in API action=abusefiltercheckmatch [[gerrit:211744]] (duration: 00m 10s) |
[production] |
| 15:31 |
<anomie> |
Synchronized php-1.26wmf5/includes/skins/SkinTemplate.php: SWAT: Revert "output mw-content-{ltr,rtl} unconditionally" [[gerrit:211893]] (duration: 00m 12s) |
[production] |
| 15:28 |
<anomie> |
Synchronized php-1.26wmf6/includes/skins/SkinTemplate.php: SWAT: Revert "output mw-content-{ltr,rtl} unconditionally" [[gerrit:211894]] (duration: 00m 13s) |
[production] |
| 15:16 |
<anomie> |
Synchronized php-1.26wmf5/includes/registration/ExtensionRegistry.php: SWAT: registration: Don't array_unique() over the queue before loading it [[gerrit:211948] (duration: 00m 12s) |
[production] |
| 15:15 |
<anomie> |
Synchronized php-1.26wmf6/includes/registration/ExtensionRegistry.php: SWAT: registration: Don't array_unique() over the queue before loading it [[gerrit:211947] (duration: 00m 12s) |
[production] |
| 14:43 |
<jynus> |
back to read/write after virt1000 database migration - migration seems ok |
[production] |
| 14:41 |
<godog> |
purge cassandra system CF metrics from graphite1001 |
[production] |
| 14:29 |
<jynus> |
temporarily going read-only for virt1000 for database migration |
[production] |
| 14:24 |
<mobrovac> |
enabled puppet on restbase1001 |
[production] |
| 14:19 |
<mobrovac> |
restbase group1 wiki keyspaces created |
[production] |
| 14:15 |
<mobrovac> |
starting manually RB with group1 wikis enabled on restbase1001 |
[production] |
| 14:11 |
<mobrovac> |
restbase100x: removed superfluous keyspaces by hand from Cassandra |
[production] |
| 13:49 |
<Steinsplitter> |
test |
[production] |
| 13:47 |
<bblack> |
done with cp40xx reboot process |
[production] |
| 13:32 |
<bblack> |
rebooting ulsfo caches (cp40xx - currently depooled from all traffic + downtimed in icinga) |
[production] |
| 13:09 |
<mobrovac> |
disabled puppet on restbase100x |
[production] |
| 12:51 |
<godog> |
bounce hhvm on mw1152 |
[production] |
| 08:26 |
<_joe_> |
restarting a few HHVM instances with a full TC space |
[production] |
| 05:05 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Tue May 19 05:03:56 UTC 2015 (duration 3m 55s) |
[production] |
| 02:46 |
<LocalisationUpdate> |
completed (1.26wmf6) at 2015-05-19 02:45:17+00:00 |
[production] |
| 02:43 |
<krinkle> |
Synchronized php-1.26wmf6/includes/resourceloader/ResourceLoader.php: Ic0df4fb5cff (duration: 00m 12s) |
[production] |
| 02:42 |
<l10nupdate> |
Synchronized php-1.26wmf6/cache/l10n: (no message) (duration: 05m 43s) |
[production] |
| 02:26 |
<LocalisationUpdate> |
completed (1.26wmf5) at 2015-05-19 02:25:05+00:00 |
[production] |
| 02:21 |
<l10nupdate> |
Synchronized php-1.26wmf5/cache/l10n: (no message) (duration: 06m 11s) |
[production] |
| 00:37 |
<ebernhardson> |
Synchronized php-1.26wmf5/includes/jobqueue/JobQueueGroup.php: Undefer push() in lazyPush() temporarily (duration: 00m 12s) |
[production] |
| 00:36 |
<ebernhardson> |
Synchronized php-1.26wmf6/includes/jobqueue/JobQueueGroup.php: Undefer push() in lazyPush() temporarily (duration: 00m 12s) |
[production] |
|
2015-05-18
§
|
| 23:49 |
<yuvipanda> |
restarted nutcracker on mw1053 and mw1107 for bd808 |
[production] |
| 23:47 |
<bd808> |
nutcracker needs restart on mw1053 and mw1107 |
[production] |
| 23:37 |
<yuvipanda> |
restarting hhvm on mw1123 |
[production] |
| 23:36 |
<ori> |
Synchronized wmf-config/CommonSettings.php: Revert "Removed "refreshLinks" from $wgJobBackoffThrottling" (duration: 00m 14s) |
[production] |
| 23:29 |
<ebernhardson> |
Synchronized wmf-config/CommonSettings.php: removed refreshlinks from #wgJobBackoffThrottling (duration: 00m 14s) |
[production] |
| 23:21 |
<hoo> |
Reverting my changes to the sites and site_identifiers tables from earlier on... apparently the export/importSites.php maintenance scripts don't work as advertised |
[production] |
| 23:03 |
<ori> |
Synchronized php-1.26wmf6/extensions/Echo: 8609cb6b90: Update Echo for cherry-picks (duration: 00m 30s) |
[production] |
| 23:02 |
<ori> |
Synchronized php-1.26wmf5/extensions/Echo: 8c619b99a6: Update Echo for cherry-picks (duration: 00m 57s) |
[production] |
| 22:46 |
<hoo> |
Updating the sites table on all wikis to reflect the language code change of bhwiki (from bh to bho). I have a backup of the old table from Wikidata in my home, should things go wrong. |
[production] |
| 20:39 |
<mforns> |
upgraded and restarted EventLogging server: 19b5b7ae719321c4b8fb112890b574051b090571 |
[production] |
| 20:12 |
<subbu> |
deployed parsoid version 8ed3e503 |
[production] |
| 19:42 |
<yurik> |
restarted graphoid service to pick up the new config https://gerrit.wikimedia.org/r/#/c/211450/ |
[production] |
| 19:35 |
<ori> |
restarted statsv on hafnium |
[production] |
| 18:29 |
<ori> |
Synchronized php-1.26wmf6/includes: 335f8a257d, e3b2255d9c (for UBN! T99468) (duration: 00m 28s) |
[production] |
| 18:28 |
<ori> |
Synchronized php-1.26wmf5/includes: 335f8a257d, e3b2255d9c (for UBN! T99468) (duration: 01m 26s) |
[production] |
| 18:06 |
<ori> |
restarted HHVM on mw1107 with libjemalloc heap profiling enabled |
[production] |
| 17:55 |
<ori> |
Enabling heap profiling on mw11107 to troubleshoot T99525 |
[production] |
| 17:08 |
<andrewbogott> |
starting all instances on labvirt1001 (well, the ones that were running before) |
[production] |