|
2015-09-11
§
|
| 16:33 |
<legoktm> |
ssh: connect to host mw1156.eqiad.wmnet port 22: Connection timed out |
[production] |
| 16:32 |
<paravoid> |
powercycling mw1156, multiple kernel backtraces in console output |
[production] |
| 16:32 |
<legoktm@tin> |
Synchronized php-1.26wmf22/resources/src/mediawiki/mediawiki.js: resourceloader: Document internal mw.loader#jobs property (duration: 01m 07s) |
[production] |
| 16:15 |
<cmjohnson1> |
mw1031 rebooting for f/w update |
[production] |
| 16:07 |
<bblack> |
enabled LRO+GRO on lvs200[123], starting pybal there again ([456] testing looks good so far) |
[production] |
| 15:44 |
<bblack> |
enabled LRO+GRO on lvs200[456] (backups). Stopping pybal on lvs200[123] to test... |
[production] |
| 15:11 |
<cmjohnson1> |
swapping pem2 cr2-eqiad |
[production] |
| 10:03 |
<jynus> |
starting nodepool in labnodepool1001 |
[production] |
| 09:21 |
<jynus> |
starting profiling of phabricator db (db1043). Very low overhead. |
[production] |
| 06:03 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Fri Sep 11 06:03:00 UTC 2015 (duration 2m 59s) |
[production] |
| 02:41 |
<l10nupdate@tin> |
LocalisationUpdate completed (1.26wmf22) at 2015-09-11 02:41:24+00:00 |
[production] |
| 02:34 |
<l10nupdate@tin> |
Synchronized php-1.26wmf22/cache/l10n: l10nupdate for 1.26wmf22 (duration: 11m 18s) |
[production] |
| 01:16 |
<ori@tin> |
Synchronized php-1.26wmf22/extensions/TitleBlacklist: 9bf13dbe0b, 3203b045f7 (duration: 00m 12s) |
[production] |
|
2015-09-10
§
|
| 23:52 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/237064/ (duration: 00m 11s) |
[production] |
| 23:47 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/237056/ (duration: 00m 11s) |
[production] |
| 23:13 |
<krenair@tin> |
Synchronized wmf-config/wikitech.php: https://gerrit.wikimedia.org/r/221825 (duration: 00m 13s) |
[production] |
| 23:04 |
<krenair@tin> |
Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/224771 (duration: 00m 12s) |
[production] |
| 21:12 |
<legoktm@tin> |
Synchronized php-1.26wmf22/extensions/Echo/modules: Align popup footer buttons to take 50% width each (duration: 00m 15s) |
[production] |
| 20:50 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: depool es1001; increase weight of es1015 and es1019 (duration: 00m 19s) |
[production] |
| 20:47 |
<ottomata> |
restarting eventlogging with 12 client side processors on eventlog1001 |
[production] |
| 20:31 |
<ottomata> |
turning off varnishncsa eventlogging eventlistener instances on frontend caches, it is now superseded by varnishkafka |
[production] |
| 20:28 |
<mutante> |
killed/restarted ganglia aggregator process for mobile-cache, upload cache, misc esams ... |
[production] |
| 20:22 |
<jynus> |
last SCAP failed on 266/466 hosts |
[production] |
| 20:21 |
<mutante> |
killed/restarted ganglia aggregator process for text-caches esams on hooft |
[production] |
| 20:17 |
<yurik> |
deployed kartotherian |
[production] |
| 20:08 |
<jynus@tin> |
Synchronized wmf-config/db-eqiad.php: Depool es1001; increase weight of es1015 and es1019 (duration: 00m 11s) |
[production] |
| 19:11 |
<twentyafterfour@tin> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedia wikis to 1.26wmf22 |
[production] |
| 19:09 |
<twentyafterfour@tin> |
Synchronized php-1.26wmf22/extensions/CentralNotice: deploy https://gerrit.wikimedia.org/r/#/c/237458/ (duration: 00m 12s) |
[production] |
| 18:57 |
<twentyafterfour> |
restarted phd on iridium |
[production] |
| 18:51 |
<twentyafterfour@tin> |
Synchronized php-1.26wmf22/extensions/Wikidata: Deploy wikidata patch: https://gerrit.wikimedia.org/r/#/c/237449/ (duration: 00m 19s) |
[production] |
| 18:23 |
<twentyafterfour@tin> |
Synchronized php-1.26wmf22: deploy https://gerrit.wikimedia.org/r/#/c/237440/ (duration: 01m 42s) |
[production] |
| 18:09 |
<cmjohnson1> |
reseating pem2 cr2-eqiad |
[production] |
| 16:52 |
<akosiaris> |
puppetswat done |
[production] |
| 16:50 |
<mobrovac> |
restbase rolling restart of rb100x |
[production] |
| 16:49 |
<mobrovac> |
restbase enabled puppet on rb100x |
[production] |
| 16:13 |
<akosiaris> |
started puppetSWAT |
[production] |
| 16:10 |
<marktraceur@tin> |
Finished scap: Make sure codfw got the last few patches sync'd to it (duration: 07m 36s) |
[production] |
| 16:03 |
<marktraceur@tin> |
Started scap: Make sure codfw got the last few patches sync'd to it |
[production] |
| 16:02 |
<marktraceur@tin> |
Synchronized php-1.26wmf22/: [SWAT] [wmf22] Revert opera redirect loop fix that caused redirect loops in Firefox (duration: 02m 30s) |
[production] |
| 15:55 |
<mobrovac> |
restbase disabled puppet on rb100x |
[production] |
| 15:45 |
<marktraceur@tin> |
Synchronized php-1.26wmf22/extensions/UploadWizard/resources/transports/mw.FormDataTransport.js: [SWAT] [wmf22] Always set 'offset' with chunked uploads, even for first chunk (offset == 0) (duration: 02m 21s) |
[production] |
| 15:26 |
<ottomata> |
started hadoop decomission of analytics1016 |
[production] |
| 15:21 |
<marktraceur@tin> |
Synchronized wmf-config/: [SWAT] Attempting another sync to mw2187 hoping it's up now (duration: 02m 22s) |
[production] |
| 15:05 |
<marktraceur@tin> |
Synchronized wmf-config/: [SWAT] [config] Beta: Enable Content Translation suggestions (duration: 02m 22s) |
[production] |
| 13:35 |
<moritzm> |
enabled ferm on mediawiki app servers in codfw |
[production] |
| 13:30 |
<jynus> |
performing schema change and maintenance on officewiki and public all wikis with flow enabled |
[production] |
| 12:51 |
<moritzm> |
enabled ferm on mediawiki API servers in codfw |
[production] |
| 12:36 |
<moritzm> |
enabled ferm on mediawiki video scalers, image scalers and job runners in codfw |
[production] |
| 09:20 |
<mobrovac> |
restbase deploying 0182962 |
[production] |
| 06:13 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Thu Sep 10 06:13:14 UTC 2015 (duration 13m 13s) |
[production] |