1051-1100 of 10000 results (31ms)
2016-03-10 ยง
21:42 <twentyafterfour@tin> Synchronized php-1.27.0-wmf.16/extensions/Wikidata: Deploy https://gerrit.wikimedia.org/r/#/c/276473/ (duration: 02m 10s) [production]
21:03 <twentyafterfour@tin> rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.16 [production]
20:32 <cmjohnson1> replacing failed disk ms1001 [production]
20:31 <AaronSchulz> Ran <<mwscriptwikiset extensions/WikimediaMaintenance/filebackend/setZoneAccess.php private.dblist --backend=local-multiwrite --private>> [production]
19:23 <godog> enable puppet in cache_cluster in eqiad, followed by the rest [production]
19:17 <godog> set timeline and math data containers as readable in swift codfw [production]
18:53 <krinkle@tin> Synchronized wmf-config/db-codfw.php: I47954e21 (duration: 00m 30s) [production]
18:46 <godog> sync wikipedia-commons-gwtoolset-metadata with swiftrepl eqiad -> codfw T129359 [production]
18:43 <yurik> deployed latest kartotherian [production]
18:19 <ori@tin> Synchronized php-1.27.0-wmf.15/includes/GlobalFunctions.php: wfShellExec() debug logging for T129467 (take 2) (duration: 00m 26s) [production]
18:13 <jynus> importing zhwiki missing records from production to labs [production]
18:13 <ori@tin> Synchronized php-1.27.0-wmf.15/includes/GlobalFunctions.php: wfShellExec() debug logging for T129467 (duration: 00m 28s) [production]
17:44 <godog> running puppet in batches on cache_upload in eqiad after https://gerrit.wikimedia.org/r/#/c/276223/ [production]
17:38 <filippo@tin> Synchronized wmf-config/filebackend-production.php: swift codfw sync replication T129089 (duration: 00m 39s) [production]
17:19 <ori> cleared hhbc bytecode repo on mw1082 and mw1245 on suspicion that old translations were reused [production]
16:36 <moritzm> mw1122: restarted hhvm [production]
15:12 <akosiaris> restart hhvm on mw1107 [production]
14:29 <moritzm> mw1107: restarted hhvm [production]
14:15 <_joe_> restarted multiple times hhvm on mw112; it didn't help, so removed manually the warmup upstart task and the old sqlite cache; also disabling puppet on the machine [production]
14:08 <urandom> increasing outbound stream throughput on restbase1002.eqiad.wmnet to 200mbps : T125842 [production]
13:32 <bblack> restarted mw1122 hhvm [production]
13:26 <bblack> mw1210: restarted hhvm [production]
13:25 <moritzm> mw1241: restarted hhvm [production]
13:20 <bblack> mw1188: restarted hhvm (before alert hit IRC, was already pending in icinga) [production]
13:19 <bblack> mw1240: restarted hhvm [production]
13:17 <bblack> mw1122: restarted hhvm, killed stale procs from Jan24 looking like: /usr/bin/hhvm --php -c /etc/hhvm/fcgi.ini -r echo ini_get("hhvm.jit_warmup_requests")?:11; [production]
13:15 <bblack> restarting hhvm on mw1112 [production]
13:12 <bblack> restarting hhvm on mw1248, mw1151 [production]
13:06 <bblack> restarting hhvm on mw1251 ... [production]
13:02 <bblack> restarting hhvm on mw1258 ... [production]
13:01 <bblack> restarting hhvm on mw1243 [production]
12:59 <bblack> restarting hhvm on mw1091 mw1249 mw1252 mw1242 ... [production]
12:56 <bblack> restarting hhvm on mw1217 [production]
12:01 <_joe_> restarted hhvm on mw1122, stuck at startup [production]
10:39 <mobrovac> restbase deploy end of 26bd4aa28 [production]
10:31 <jynus> setting up the rest of the cross-datacenter master-master connections pending in wmf databases [production]
10:09 <godog> decommissioning restbase1002.eqiad.wmnet : T125842 [production]
09:50 <volans@tin> Synchronized wmf-config/db-codfw.php: Rebalance external storage servers in codfw T127330 (duration: 00m 34s) [production]
09:50 <mobrovac> restbase deploy start of 26bd4aa28 on restbase1001 [production]
08:55 <ema> apache2 and hhvm restarted on mw1107, mw1122 and mw1119 [production]
03:08 <l10nupdate@tin> ResourceLoader cache refresh completed at Thu Mar 10 03:08:54 UTC 2016 (duration 9m 38s) [production]
02:59 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.16) (duration: 12m 33s) [production]
02:33 <mwdeploy@tin> sync-l10n completed (1.27.0-wmf.15) (duration: 14m 08s) [production]
00:45 <catrope@tin> Synchronized php-1.27.0-wmf.15/extensions/ZeroBanner: SWAT (duration: 00m 28s) [production]
00:44 <catrope@tin> Synchronized php-1.27.0-wmf.15/extensions/MobileFrontend: SWAT (duration: 00m 32s) [production]
00:39 <catrope@tin> Synchronized php-1.27.0-wmf.16/extensions/ZeroBanner: SWAT (duration: 00m 28s) [production]
00:39 <catrope@tin> Synchronized php-1.27.0-wmf.16/extensions/MobileFrontend: SWAT (duration: 00m 36s) [production]
00:30 <ori> Upgrade of HHVM package to 3.12.1+dfsg-1 complete on all eqiad hosts save terbium [production]
00:23 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Set NS_PROJECT on azwiktionary (duration: 00m 29s) [production]
00:13 <catrope@tin> Synchronized wmf-config/CommonSettings.php: Remove svnadmins group (duration: 00m 41s) [production]