|
2015-05-20
§
|
| 15:06 |
<springle> |
db1045 pt-osc reindexing (should be low load, ~2hr) |
[production] |
| 14:36 |
<aude> |
Synchronized wmf-config/InitialiseSettings.php: Enable usage tracking on itwiki and wikiquote (duration: 00m 16s) |
[production] |
| 14:25 |
<milimetric> |
Deployed Event Logging Server with better batch insertion on Monday, May 18 (apologies for late notice) |
[production] |
| 13:13 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1045; depool db1026 (duration: 00m 13s) |
[production] |
| 10:19 |
<jynus> |
Synchronized wmf-config/db-eqiad.php: repool db1063 (duration: 00m 11s) |
[production] |
| 09:43 |
<_joe_> |
stopping puppet, fiddling with HHVM parameters on mw1114 |
[production] |
| 09:37 |
<Coren> |
tools kicked grrrit-wm in the diodes. |
[production] |
| 09:35 |
<jynus> |
Synchronized wmf-config/db-eqiad.php: repool db1063 (duration: 00m 12s) |
[production] |
| 06:45 |
<jynus> |
Synchronized wmf-config/db-eqiad.php: depool db1063 for maintenance (duration: 00m 11s) |
[production] |
| 06:43 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Wed May 20 06:42:22 UTC 2015 (duration 42m 21s) |
[production] |
| 03:13 |
<LocalisationUpdate> |
completed (1.26wmf6) at 2015-05-20 03:12:31+00:00 |
[production] |
| 03:06 |
<l10nupdate> |
Synchronized php-1.26wmf6/cache/l10n: (no message) (duration: 09m 40s) |
[production] |
| 02:41 |
<LocalisationUpdate> |
completed (1.26wmf5) at 2015-05-20 02:40:07+00:00 |
[production] |
| 02:36 |
<l10nupdate> |
Synchronized php-1.26wmf5/cache/l10n: (no message) (duration: 06m 30s) |
[production] |
| 01:14 |
<springle> |
Synchronized wmf-config/db-eqiad.php: depool db1045 (duration: 00m 15s) |
[production] |
| 00:43 |
<ebernhardson> |
Synchronized wmf-config/: Per-user poolcounter triggered many more times than expected (duration: 00m 15s) |
[production] |
| 00:42 |
<ebernhardson> |
Synchronized wmf-config/PoolCounterSettings-common.php: Enable per-user poolcounter in CirrusSearch on all wikis (duration: 00m 14s) |
[production] |
| 00:41 |
<ebernhardson> |
Synchronized wmf-config/InitialiseSettings.php: Enable per-user poolcounter in CirrusSearch on all wikis (duration: 00m 12s) |
[production] |
| 00:40 |
<ebernhardson> |
Synchronized php-1.26wmf5/extensions/NavigationTiming/: Update NavigationTiming for cherry-picks in 1.26wmf5 (duration: 00m 12s) |
[production] |
| 00:39 |
<ebernhardson> |
Synchronized php-1.26wmf6/extensions/NavigationTiming/: Update NavigationTiming for cherry-picks in 1.26wmf6 (duration: 00m 12s) |
[production] |
| 00:36 |
<ebernhardson> |
Synchronized php-1.26wmf5/extensions/CirrusSearch/: Bump CirrusSearch in 1.26wmf5 for poolcounter error message updates (duration: 00m 11s) |
[production] |
| 00:35 |
<ebernhardson> |
Synchronized php-1.26wmf6/extensions/CirrusSearch/: Bump CirrusSearch in 1.26wmf6 for poolcounter error message updates (duration: 00m 13s) |
[production] |
| 00:34 |
<ebernhardson> |
Synchronized php-1.26wmf5/extensions/CirrusSearch/: Bump CirrusSearch in 1.26wmf5 for poolcounter error message updates (duration: 00m 12s) |
[production] |
| 00:32 |
<ebernhardson> |
Synchronized php-1.26wmf6/extensions/CirrusSearch/: Bump CirrusSearch in 1.26wmf6 for poolcounter error message updates (duration: 00m 12s) |
[production] |
|
2015-05-19
§
|
| 23:35 |
<gwicke> |
deployed RESTBase 90817c2a |
[production] |
| 23:20 |
<bd808> |
Synchronized wmf-config/InitialiseSettings.php: logstash: Exclude jobrunner debug messages (duration: 00m 12s) |
[production] |
| 23:10 |
<bd808> |
Synchronized wmf-config/InitialiseSettings.php: Enable NewUserMessage on maiwiki and pawiki (duration: 00m 12s) |
[production] |
| 22:06 |
<ejegg> |
updated payment from e89d18ee20abcb1ca3c455e6a298bf8a6aa84442 to 858b87319daa3d66f62eb32e08cefc6b061748d1 |
[production] |
| 21:16 |
<kaldari> |
Synchronized php-1.26wmf6/extensions/MobileFrontend: syncing MobileFrontend for 1.26wmf6 (duration: 00m 11s) |
[production] |
| 21:15 |
<kaldari> |
Synchronized php-1.26wmf6/extensions/Gather: syncing Gather for 1.26wmf6 (duration: 00m 12s) |
[production] |
| 21:07 |
<robh> |
merging fixes to sodium, mailing list outage fixed |
[production] |
| 20:51 |
<andrewbogott> |
rebooting/reimaging virt1005, virt1006, 1007 |
[production] |
| 20:22 |
<mutante> |
mailman: killed processes by user "list". started mailman |
[production] |
| 19:40 |
<ori> |
Synchronized wmf-config/CommonSettings.php: Ia6a2cb7: Removed "refreshLinks" from $wgJobBackoffThrottling (duration: 00m 12s) |
[production] |
| 19:37 |
<anomie> |
Finished scap: Step 2 for deploying ApiFeatureUsage: sync the config, and l10n data again because I don't think it did last time (duration: 44m 34s) |
[production] |
| 19:25 |
<robh> |
mailman permission errors abound! had to take it offline again and fixing |
[production] |
| 19:02 |
<robh> |
mailman is back to routing mail normally (still testing rename parts) |
[production] |
| 18:53 |
<anomie> |
Started scap: Step 2 for deploying ApiFeatureUsage: sync the config, and l10n data again because I don't think it did last time |
[production] |
| 18:51 |
<anomie> |
Finished scap: Step 1 for deploying ApiFeatureUsage: sync the code and l10n data (duration: 05m 39s) |
[production] |
| 18:46 |
<anomie> |
Started scap: Step 1 for deploying ApiFeatureUsage: sync the code and l10n data |
[production] |
| 18:39 |
<yuvipanda> |
issuing start command for all hosts on labvirt1006, just to make sure |
[production] |
| 18:37 |
<yuvipanda> |
labvirt1006 rebooting, long POST |
[production] |
| 18:32 |
<yuvipanda> |
restarted labvirt1006 |
[production] |
| 18:21 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group1 to 1.26wmf6 |
[production] |
| 18:17 |
<robh> |
stopping mailman again for further planned work T99098 |
[production] |
| 17:44 |
<robh> |
mailing lists still down, scrubbing list archives is painful and error prone |
[production] |
| 17:34 |
<ottomata> |
starting reboots of analytics worker nodes in order to enable hyperthreading Bug: https://phabricator.wikimedia.org/T90640 |
[production] |
| 17:06 |
<robh> |
puppet stopped on sodium (dont need it restarting mailman while im working) |
[production] |
| 17:05 |
<robh> |
starting mailman downtime window to scrub content off list archive per T99098 |
[production] |
| 17:00 |
<bblack> |
automated reboots of esams/eqiad non-upload caches starting (should auto-downtime, should be no real impact)... |
[production] |