5351-5400 of 10000 results (43ms)
2016-05-25 §
09:02 <akosiaris> upload php5_5.3.10-1ubuntu3.23+wmf1 on apt.wikimedia.org/precise-wikimedia [production]
08:39 <jynus> perform second schema change on x1 wikis for T135699 [production]
08:27 <moritzm> restarting apache on uranium [production]
08:08 <moritzm> installing php5 security updates on trusty systems [production]
07:41 <jynus> rm runJobs.log-20160[1-3]*.gz on fluorine archive log [production]
02:00 <mutante> reboot unresponse mw1140 [production]
01:48 <dereckson@tin> scap sync-l10n completed (1.28.0-wmf.2) (duration: 15m 24s) [production]
00:22 <dereckson@tin> Synchronized /srv/mediawiki-staging/php-1.28.0-wmf.3/extensions/Flow/maintenance/FlowRemoveOldTopics.php: Don't assume workflows/revisions are inserted in chronological order (T119509) (duration: 00m 23s) [production]
00:16 <dereckson@tin> Synchronized /srv/mediawiki-staging/php-1.28.0-wmf.2/extensions/Flow/maintenance/FlowRemoveOldTopics.php: Don't assume workflows/revisions are inserted in chronological order (T119509) (duration: 00m 28s) [production]
2016-05-24 §
23:58 <dereckson@tin> Synchronized php-1.28.0-wmf.2/extensions/MobileFrontend/tests/phpunit/MobileFormatterTest.php: "Pi" article on mobile en.wp throws a 503 fatal (no-op, [[Gerrit:290587]]) (duration: 00m 23s) [production]
23:58 <dereckson@tin> Synchronized php-1.28.0-wmf.2/extensions/MobileFrontend/includes/MobileFormatter.php: "Pi" article on mobile en.wp throws a 503 fatal (T135923, [[Gerrit:290587]] + [[Gerrit:290598]]) (duration: 00m 24s) [production]
23:46 <Dereckson> scap pull on mw1140 (duration: 02m 42s) [production]
23:20 <dereckson@tin> Synchronized wmf-config/InitialiseSettings.php: Set Tamil projects to use uca-ta collation (T75453) (duration: 02m 18s) [production]
23:07 <dereckson@tin> Synchronized wmf-config/InitialiseSettings.php: Adding WMF Support and Safety user groups to meta (T136046) (duration: 00m 26s) [production]
22:29 <twentyafterfour@tin> rebuilt wikiversions.php and synchronized wikiversions files: group0 to 1.28.0-wmf.3 [production]
22:24 <twentyafterfour@tin> scap sync-l10n completed (1.28.0-wmf.3) (duration: 00m 37s) [production]
22:21 <twentyafterfour@tin> Finished scap: sync testwiki to wmf/1.28.0-wmf.3 (duration: 41m 48s) [production]
21:39 <twentyafterfour@tin> Started scap: sync testwiki to wmf/1.28.0-wmf.3 [production]
21:24 <volans> set innodb_flush_log_at_trx_commit=0 on db1056 that is lagging behind as a temporary measure [production]
21:13 <Krinkle> mwscript deleteEqualMessages.php --wiki mrwiki (T45917) [production]
21:08 <mutante> mw1137,mw1146 restarted hhvm service [production]
20:34 <mutante> ocg1003 - reinstalled, replaced puppet cert, salt key..re-added [production]
20:07 <Krinkle> mwscript deleteEqualMessages.php --wiki diqwiki (T45917) [production]
19:55 <Krinkle> mwscript deleteEqualMessages.php --wiki cywiktionary (T45917) [production]
19:50 <mutante> mw1133 - Could not allocate memory, restarted hhvm service, ran puppet [production]
19:48 <mutante> mw1142 - Could allocate memory on puppet run, restarted hhvm service, that fixed it [production]
19:44 <Krinkle> mwscript deleteEqualMessages.php --wiki warwiki (T45917) [production]
19:14 <mutante> ocg1003, powercycle for reinstall, scheduled downtime [production]
18:28 <urandom> Starting bootstrap of restbase1010-c.eqiad.wmnet : T134016 [production]
18:08 <urandom> Starting bootstrap of restbase2006-b.codfw.wmnet : T95253 [production]
17:38 <elukey> soft rebooted mw1134 due to unresponsiveness (no root login, no ssh login, memory exhaustion from server-board) [production]
17:22 <yurik> deployed & restarted kartotherian & tilerator. https://gerrit.wikimedia.org/r/#/c/290494/ https://gerrit.wikimedia.org/r/#/c/290497/ [production]
16:11 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:290475|Enable Compact Language Links as default in Beta]] PART II (duration: 00m 25s) [production]
16:10 <thcipriani@tin> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:290475|Enable Compact Language Links as default in Beta]] PART I (duration: 00m 23s) [production]
16:05 <Krinkle> mwscript deleteEqualMessages.php --wiki fiwikinews (T45917) [production]
15:57 <thcipriani@tin> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:290165|Enable Compact Language Links as default in Beta]] PART II (duration: 00m 25s) [production]
15:57 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:290165|Enable Compact Language Links as default in Beta]] PART I (duration: 00m 23s) [production]
15:26 <_joe_> shutting down mw2001-2060 [production]
15:11 <thcipriani@tin> Synchronized wmf-config/Wikibase.php: SWAT: [[gerrit:290466|Make entityNamespaces setting available to Wikibase Client wikis]] (duration: 00m 29s) [production]
15:01 <_joe_> revoking certs, puppet facts, salt keys for mw2001-60 [production]
14:01 <jynus> dropping labswiki.bounce_records on db1-BETA [production]
13:29 <elukey> restarted hhvm on mw1145 (ran hhvm-dump-debug, output available in hhvm.14155.bt) [production]
13:23 <moritzm> reverted net.netfilter.nf_conntrack_tcp_timeout_time_wait on kafka1013 back to 65 (as it should have been set by sysctl.d) [production]
12:49 <elukey> stopping kafka on kafka1013 and rebooting the host for kernel upgrade [production]
12:32 <jynus> deploying GTID to all codfw db production hosts [production]
12:10 <_joe_> cleaning puppet facts, salt keys for mw2001-2060 [production]
11:53 <mobrovac> restbase resting nodes to pick up https://gerrit.wikimedia.org/r/#/c/290264/ [production]
11:22 <jynus> enabling GTID on s1 codfw db servers [production]
11:21 <moritzm> rolling restart of elasticsearch in logstash cluster to pick up openjdk security update [production]
10:50 <_joe_> disabling puppet on mw2001-60 (minus 2017) for decommissioning [production]