9251-9300 of 10000 results (82ms)
2017-06-14 §
18:50 <otto@tin> Finished deploy [eventlogging/analytics@1ce446d]: (no justification provided) (duration: 00m 04s) [production]
18:49 <otto@tin> Started deploy [eventlogging/analytics@1ce446d]: (no justification provided) [production]
18:34 <niharika29@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/358007/ Add wmgBabelMainCategory for many languages (duration: 00m 43s) [production]
18:32 <niharika29@tin> scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) [production]
18:25 <niharika29@tin> Synchronized wmf-config/InitialiseSettings.php: Sort wmgBabelMainCategory alphabetically https://gerrit.wikimedia.org/r/#/c/358006/ (duration: 00m 44s) [production]
18:24 <jynus> reimporting data from pc1004 to db1096 [production]
18:17 <niharika29@tin> Synchronized tests/cirrusTest.php: https://gerrit.wikimedia.org/r/#/c/358625/ Test elastic2020 does not fall out of cluster (duration: 00m 43s) [production]
18:13 <niharika29@tin> Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/358625/ Test elastic2020 does not fall out of cluster (duration: 00m 44s) [production]
18:06 <moritzm> installing unzip security updates [production]
17:55 <moritzm> restarting hhvm on mw1261-mw1265 to pick up libxslt update [production]
17:49 <moritzm> installing mongodb update from jessie point release on tungsten [production]
16:03 <godog> point varnish upload in esams back to eqiad [production]
16:00 <mobrovac@tin> Finished deploy [restbase/deploy@4c1cdd0]: (no justification provided) (duration: 04m 51s) [production]
15:55 <mobrovac@tin> Started deploy [restbase/deploy@4c1cdd0]: (no justification provided) [production]
15:44 <godog> point varnish upload back to swift eqiad [production]
15:14 <ema> restart varnish-backend on cp2017 [production]
15:08 <moritzm> installing systemd bugfix updates from jessie point update [production]
15:00 <ema> restart varnish-backend on cp2014 [production]
13:50 <zeljkof> eu swat finished [production]
13:42 <zfilipin@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:358929|Remove ContentTranslationTargetNamespace config (T167865)]] (duration: 00m 43s) [production]
13:41 <zfilipin@tin> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:358958|Remove unneeded ContentTranslationTargetNamespace (T167865)]] (duration: 00m 44s) [production]
13:35 <zfilipin@tin> scap failed: average error rate on 3/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) [production]
12:24 <jynus@tin> Synchronized wmf-config/db-codfw.php: Switchover pc2004 to db2072 (duration: 00m 43s) [production]
12:13 <akosiaris> upload apertium-spa-ita_0.2.0~r78826-1+wmf to apt.wikimedia.org/jessie-wikimedia/main [production]
12:13 <akosiaris> upload apertium-fra-cat_1.2.0~r78602-1+wmf to apt.wikimedia.org/jessie-wikimedia/main [production]
11:41 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Switchover pc1004 to db1096 (duration: 00m 54s) [production]
11:34 <jynus> about to deploy performance-impacting change on the parsercache persistent storage T167567 [production]
11:19 <marostegui> Deploy alter table s4 - labsdb1011 - T166206 [production]
09:46 <marostegui> Rename table titlekey before dropping it on enwiki - db1089 - T164949 [production]
09:18 <godog> delete files older than 365d from 'servers' graphite hierarchy [production]
07:59 <marostegui> Drop table updates on s3 - T139342 [production]
07:32 <moritzm> installing zziplib security updates on jessie [production]
07:04 <elukey> restart pdfrender on scb200[2,4] (xpra race condition) [production]
07:03 <elukey> restart pdfrender on scb1004 (xpra race condition) [production]
06:32 <moritzm> installing remaining libtasn security updates [production]
03:14 <l10nupdate@tin> ResourceLoader cache refresh completed at Wed Jun 14 03:14:28 UTC 2017 (duration 6m 56s) [production]
03:07 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.5) (duration: 14m 52s) [production]
02:32 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.4) (duration: 07m 58s) [production]
01:48 <mutante> netmon1002 - chown rancid:rancid /var/lib/rancid ; touch /var/lib/rancid/.gitconfig, let rancid write to config, then git config --global user.email and user.name as the rancid user | fix permissions on .git/objects files, let rancid user own them all | re-commit .gitingore change | SSH_AUTH_SOCK=/run/keyholder/proxy.sock /usr/lib/rancid/bin/rancid-run as user "rancid" runs clean, [production]
01:20 <mutante> netmon1002 - copied missing router.db, routers.all/.down/.up over from netmon1001 to /var/lib/rancid/core. routers.db is an untracked file, the others are in .gitignore. this is all like on netmon1001 as well. adding routers.db to .gitignore file on both, like the other router* files already were (T159756) [production]
01:00 <mutante> netmon1002 - locally "git clone /var/lib/rancid/GIT/core" into /var/lib/rancid (i rsynced that but it's a bare repository without a work tree. work tree is /var/lib/rancid/core (after this) (T159756) [production]
00:44 <mutante> naos: disarm keyholder and armed it again to proof i didn't break anything on jessie by fixing keyholder on stretch with gerrit:358884 [production]
00:39 <demon@tin> Synchronized wmf-config/CommonSettings.php: extdist update (duration: 00m 44s) [production]
00:09 <aaron@tin> Synchronized wmf-config/InitialiseSettings.php: Capture messages on 'autoloader' debug log channel (duration: 00m 44s) [production]
2017-06-13 §
23:29 <RainbowSprinkles> gerrit: upgrading on master 2.13.4-13-gc0c5cc4742 -> 2.13.8-1-g7c438d37a2 (been running on slave for a week) [production]
23:13 <mutante> contint1001 - started zuul using the old init script [production]
23:04 <mutante> netmon1001/1002: rsynced /var/lib/rancid/CVS and /var/lib/rancid/GIT from 1001 to 1002 for rancid migration (T159756) [production]
23:04 <demon@tin> Synchronized php-1.30.0-wmf.4/extensions/OpenStackManager: Re-adding deleted special page (duration: 00m 45s) [production]
22:06 <ejegg> updated fundraising tools from f2522cdabf1741a60b7b60ac8f7ead7afd50b054 to 585f546aa0c092ccd938a9a01b4bc3eb7662804d [production]
21:59 <gwicke> restarted pdfrender on scb1003; was spinning on CPU & using 15G of memory (!) [production]