7551-7600 of 10000 results (61ms)
2018-05-16 §
08:54 <jynus> restart dbproxy1002 for upgrade [production]
08:48 <moritzm> upgrading mw1221-mw1235 to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 [production]
08:45 <jynus> restart dbproxy1003 for upgrade [production]
08:20 <marostegui> Stop MySQL on db1120 on s2, s4, s6 and s7 to copy its content to db2075 - T190704 [production]
07:57 <_joe_> depooled ULSFO from live traffic in dns [production]
07:41 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Depool db2075 to convert it to temporary sanitarium (duration: 01m 20s) [production]
07:36 <moritzm> installing systemd updates from stretch SUA [production]
07:33 <marostegui> Deploy schema change on db1094 - T191519 T188299 T190148 [production]
07:33 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1094 for alter table (duration: 01m 20s) [production]
07:27 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Repool db1086 after alter table (duration: 01m 34s) [production]
07:25 <marostegui> Drop unused tables msg_resource msg_resource_links from s1 - T194663 [production]
07:18 <moritzm> upgrading mw1240-mw1258 to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 [production]
06:56 <moritzm> upgrading mwdebug servers to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 [production]
06:45 <marostegui> Drop unused tables msg_resource msg_resource_links from s3 codfw - T194663 [production]
06:25 <marostegui> Drop unused tables msg_resource msg_resource_links from s7 - T194663 [production]
06:19 <marostegui> Drop unused tables msg_resource msg_resource_links from s4 - T194663 [production]
06:19 <elukey> update analytics-in4 on cr1/cr2 eqiad to allow conf100[4-6] (new zookeeper hosts) [production]
06:17 <marostegui> Drop unused tables msg_resource msg_resource_links from s8 - T194663 [production]
06:15 <marostegui> Drop unused tables msg_resource msg_resource_links from s6 - T194663 [production]
06:11 <marostegui> Drop unused tables msg_resource msg_resource_links from s5 - T194663 [production]
05:54 <elukey> removed acpi_power_meter manually from conf1004 (blacklisted module in puppet), Acpi errors in dmesg [production]
05:22 <marostegui> Deploy schema change on db1086 - T191519 T188299 T190148 [production]
05:21 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Depool db1086 for alter table (duration: 01m 23s) [production]
05:14 <marostegui> Deploy schema change on s2 primary master db1054 - T191519 T188299 T190148 [production]
04:43 <kartik@tin> Finished deploy [cxserver/deploy@7e898c7]: Update cxserver to 112a1a1 (T191285) (duration: 03m 52s) [production]
04:39 <kartik@tin> Started deploy [cxserver/deploy@7e898c7]: Update cxserver to 112a1a1 (T191285) [production]
03:46 <l10nupdate@tin> ResourceLoader cache refresh completed at Wed May 16 03:46:12 UTC 2018 (duration 7m 41s) [production]
03:38 <l10nupdate@tin> scap sync-l10n completed (1.32.0-wmf.4) (duration: 16m 26s) [production]
02:40 <l10nupdate@tin> scap sync-l10n completed (1.32.0-wmf.3) (duration: 08m 25s) [production]
2018-05-15 §
23:33 <mutante> creating ganeti VM webperf2002.eqiad.wmnet on ganeti2004 (link: private, row: A, cpus: 4, ram: 8, disk: 50) (T194390) [production]
23:11 <dzahn@neodymium> conftool action : set/pooled=yes; selector: name=mw2139.codfw.wmnet [production]
23:10 <mutante> mw2139 - reimaged, scap pull, apache-fast-test baseurls from naos, repooled with confctl (T194426) [production]
23:06 <mutante> creating ganeti VM webperf1002.eqiad.wmnet on ganeti1004 (link: private, row: A, cpus: 4, ram: 8, disk: 50) (T194390) [production]
22:39 <samwilson@tin> Synchronized wmf-config/InitialiseSettings.php: Deploying GlobalPreferences T190425 (duration: 01m 21s) [production]
22:21 <twentyafterfour@tin> rebuilt and synchronized wikiversions files: group0 to 1.32.0-wmf.4 refs T191050 [production]
22:18 <twentyafterfour@tin> Synchronized php-1.32.0-wmf.4/includes/api/ApiLogin.php: fix syntax error (duration: 01m 39s) [production]
21:34 <twentyafterfour@tin> Finished scap: testwikis to 1.32.0-wmf.4 refs T191050 (duration: 66m 19s) [production]
20:27 <twentyafterfour@tin> Started scap: testwikis to 1.32.0-wmf.4 refs T191050 [production]
20:22 <mutante> [radium:~] $ sudo apt-get autoremove [production]
19:53 <cwd> re-enabled process-control [production]
19:42 <twentyafterfour@tin> scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_770814178" --threads=10 --lang en --quiet' returned non-zero exit status 255 (duration: 02m 47s) [production]
19:40 <cwd> disabled process-control for sql trigger refresh [production]
19:39 <twentyafterfour@tin> Started scap: testwikis wikis to 1.32.0-wmf.4 [production]
19:05 <mutante> mw2139 - wmf-auto-reimage --conftoool --new (because it got "Failed to icinga_downtime" and has a new mainboard (T194426) [production]
19:03 <mutante> mw2139 - wmf-auto-reimage --conftoool --no-verify (T194426) [production]
17:27 <twentyafterfour> branching 1.32.0-wmf.4 refs T191050 [production]
17:15 <ottomata> rolling restart kafka-jumbo100[456] [production]
16:36 <elukey> rolling restart of aqs on aqs* nodes to pick up the new druid config [production]
16:27 <milimetric@tin> Finished deploy [analytics/refinery@679cf09]: Update partition drop script after schema change (duration: 07m 13s) [production]
16:25 <mobrovac@tin> Started restart [cpjobqueue/deploy@58935d5]: (no justification provided) [production]