|
2015-05-07
§
|
| 12:30 |
<bblack> |
rebooting cp1070 |
[production] |
| 12:26 |
<godog> |
bounce uwsgi on graphite1001 |
[production] |
| 12:25 |
<godog> |
bounce uwsgi on graphite1001 |
[production] |
| 10:26 |
<godog> |
bounce uwsgi on graphite1001 |
[production] |
| 10:01 |
<mark> |
Decreased labstore1001 md125 sync_speed_min from 80000 to 40000 |
[production] |
| 09:35 |
<mark> |
Increased /sys/block/md125/md/sync_speed_min from 4000 to 40000 |
[production] |
| 09:29 |
<mark> |
Increased /sys/block/md125/md/sync_speed_min from 1000 to 4000 |
[production] |
| 05:40 |
<LocalisationUpdate> |
ResourceLoader cache refresh completed at Thu May 7 05:39:36 UTC 2015 (duration 39m 35s) |
[production] |
| 03:03 |
<LocalisationUpdate> |
completed (1.26wmf5) at 2015-05-07 03:02:50+00:00 |
[production] |
| 02:59 |
<l10nupdate> |
Synchronized php-1.26wmf5/cache/l10n: (no message) (duration: 08m 35s) |
[production] |
| 02:36 |
<LocalisationUpdate> |
completed (1.26wmf4) at 2015-05-07 02:35:43+00:00 |
[production] |
| 02:35 |
<springle> |
Synchronized wmf-config/db-eqiad.php: repool db1054 in s2, warm up (duration: 01m 09s) |
[production] |
| 02:29 |
<l10nupdate> |
Synchronized php-1.26wmf4/cache/l10n: (no message) (duration: 09m 27s) |
[production] |
| 02:14 |
<krenair> |
Synchronized wmf-config: update interwiki.cdb, T98429 (duration: 00m 24s) |
[production] |
| 01:50 |
<bblack> |
we're still hitting cap on Zayo as of shortly-ago in graphs and seeing smokeping loss, moved california to eqiad |
[production] |
| 00:13 |
<mutante> |
running refreshLinks.php for s2 |
[production] |
| 00:11 |
<catrope> |
Synchronized php-1.26wmf4/extensions/MobileFrontend/: SWAT (duration: 00m 42s) |
[production] |
| 00:11 |
<gwicke> |
deployed RESTBase 8865b9c48 |
[production] |
|
2015-05-06
§
|
| 23:43 |
<catrope> |
Synchronized php-1.26wmf5/extensions/VisualEditor: SWAT (duration: 00m 18s) |
[production] |
| 23:43 |
<catrope> |
Synchronized php-1.26wmf5/extensions/MobileFrontend: SWAT (duration: 00m 34s) |
[production] |
| 23:19 |
<RoanKattouw> |
Running populateShortUrl.phg on knwiki |
[production] |
| 23:16 |
<RoanKattouw> |
Running namespaceDupes.php on tewikiquote |
[production] |
| 23:15 |
<catrope> |
Synchronized wmf-config/InitialiseSettings.php: SWAT (duration: 00m 17s) |
[production] |
| 23:12 |
<RoanKattouw> |
Created shorturls table on knwiki |
[production] |
| 20:39 |
<twentyafterfour> |
Purged l10n cache for 1.26wmf3 |
[production] |
| 20:37 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.26wmf5 |
[production] |
| 20:32 |
<twentyafterfour> |
rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.26wmf4 |
[production] |
| 20:29 |
<apergos> |
salt upgraded to 2014.7.5 on all precise/trusty/jessie hosts in production except for: labcontrol2001, tin, virt1000 (deferred) and dysprosium/labvirt1005/labstore1002 (down) |
[production] |
| 20:15 |
<twentyafterfour> |
Synchronized php-1.26wmf5/extensions/MobileFrontend/javascripts/modules/search/init.js: Temporarily disable MobileWebSearch logging (duration: 00m 36s) |
[production] |
| 20:14 |
<twentyafterfour> |
ignore all rumors of scap failures, the scaps were successful, with the exception of snapshot1004.eqiad.wmnet which hangs every time |
[production] |
| 20:14 |
<twentyafterfour> |
Synchronized php-1.26wmf4/extensions/MobileFrontend/javascripts/modules/search/init.js: Temporarily disable MobileWebSearch logging (duration: 00m 37s) |
[production] |
| 20:12 |
<twentyafterfour> |
scap failed: OSError [Errno 2] No such file or directory: '/var/lock/scap' (duration: 27m 49s) |
[production] |
| 19:44 |
<twentyafterfour> |
Started scap: testwiki to php-1.26wmf5 and rebuild l10n cache |
[production] |
| 18:39 |
<mutante> |
restarting apache on rhodium |
[production] |
| 18:34 |
<bblack> |
rebooting cp3030 |
[production] |
| 18:14 |
<andrewbogott> |
restarted gmetad on uranium |
[production] |
| 17:41 |
<andrewbogott> |
powering down virt1005 and virt1006 |
[production] |
| 17:38 |
<andrewbogott> |
depuppeting and decommissioning virt1005 and virt1006 |
[production] |
| 17:24 |
<aude> |
Synchronized wmf-config/InitialiseSettings.php: Enable Wikibase usage tracking on enwikivoyage, fawiki and hewiki (duration: 00m 18s) |
[production] |
| 17:03 |
<jgage> |
hadoop active namenode switched back to analytics1001 after rack C4 switch replacement |
[production] |
| 16:43 |
<apergos> |
done with all trusty salt updates in pro except for labcontrol1002 (?), doing jessie now in very tiny batches, it's being trouble |
[production] |
| 15:29 |
<bd808> |
Stashed uncommitted change to scap on tin that disabled php opening tag check for sync-file |
[production] |
| 15:27 |
<bd808> |
Updated scap to 57036d2 (Update statsd events) |
[production] |
| 15:27 |
<bd808> |
trebuchet checkout for scap/scap failed for mw1113.eqiad.wmnet, mw1222.eqiad.wmnet, mw1104.eqiad.wmnet |
[production] |
| 15:25 |
<bd808> |
trebuchet fetch for scap/scap failed on mw1222.eqiad.wmnet |
[production] |
| 15:04 |
<bd808> |
Synchronized wmf-config/InitialiseSettings.php: Send group0 + group1 MediaWiki events to logstash {{gerrit|209170}} (duration: 00m 16s) |
[production] |
| 14:32 |
<cmjohnson1> |
shutting down db1054 for maintenance |
[production] |
| 14:22 |
<_joe_> |
depooling the HHVM imagescaler |
[production] |
| 14:20 |
<Nemo_bis> |
phabricator went down again for some minutes, seems ok now? |
[production] |
| 14:17 |
<_joe_> |
pooling the HHVM imagescalers to test if the issues are solved now. |
[production] |