|
2019-04-10
ยง
|
| 11:46 |
<dcausse> |
elastisearch search cluster: reindexing zh-min-nan wikis (T219533) |
[production] |
| 11:42 |
<arturo> |
force deleted job 1262132 because it was stucked (trying to depool exec node for T218126) |
[tools.musikbot] |
| 11:23 |
<arturo> |
T218126 hard reboot tools-sgeexec-0940 |
[tools] |
| 11:03 |
<arturo> |
T218126 hard reboot tools-sgeexec-0928 |
[tools] |
| 10:55 |
<moritzm> |
upgrading nodejs on analytics-tool1002 to latest node 10 version from component/node10 |
[production] |
| 10:53 |
<arturo> |
force deleted job 1264656 because it was stucked (trying to depool exec node for T218126) |
[tools.ukbot] |
| 10:49 |
<arturo> |
T218126 hard reboot tools-sgeexec-0923 |
[tools] |
| 10:46 |
<gilles> |
T220265 setZoneAccess on all wikis finished |
[production] |
| 10:43 |
<arturo> |
T218126 hard reboot tools-sgeexec-0915 |
[tools] |
| 10:42 |
<arturo> |
force deleted job 853604 because it was stucked (trying to depool exec node for T218126) |
[tools.himo] |
| 10:40 |
<akosiaris> |
upgrade kubernetes-node on kubestage1002 (staging cluster) to 1.12.7-1 T220405 |
[production] |
| 10:33 |
<moritzm> |
upgrading nodejs on aqs* to latest node 10 version from component/node10 |
[production] |
| 10:27 |
<arturo> |
T218126 hard reboot tools-sgeexec-0935 |
[tools] |
| 10:26 |
<arturo> |
force deleted job 1739159 because it was stucked (trying to depool exec node for T218126) |
[tools.jarbot] |
| 10:26 |
<arturo> |
force deleted job 1739135 because it was stucked (trying to depool exec node for T218126) |
[tools.aivanalysis] |
| 10:25 |
<fsero> |
resizing disk on mwdebug2001 T219989 |
[production] |
| 10:19 |
<arturo> |
T218126 hard reboot tools-sgeexec-0914 |
[tools] |
| 10:17 |
<akosiaris> |
upload kubernetes_1.12.7-1 to apt.wikimedia.org/stretch-wikimedia component main T220405 |
[production] |
| 10:16 |
<arturo> |
force deleted job 1045059 because it was stucked (trying to depool exec node for T218126) |
[tools.jarbot] |
| 10:15 |
<arturo> |
force deleted job 1044941 because it was stucked (trying to depool exec node for T218126) |
[tools.listeria] |
| 10:14 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1064 T217453 (duration: 00m 59s) |
[production] |
| 10:08 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1120 T217453 (duration: 01m 03s) |
[production] |
| 10:01 |
<arturo> |
T218126 hard reboot tools-sgeexec-0907 |
[tools] |
| 09:59 |
<moritzm> |
upgrading labweb hosts (wikitech) to HHVM 3.18.5+dfsg-1+wmf8+deb9u2 and wikidiff 1.8.1 (T203069) |
[production] |
| 09:58 |
<arturo> |
force deleted job 871945 because it was stucked (trying to depool exec node for T218126) |
[tools.citationhunt] |
| 09:57 |
<arturo> |
force deleted job 853968 because it was stucked (trying to depool exec node for T218126) |
[tools.wmcounter] |
| 09:56 |
<arturo> |
force deleted job 853139 because it was stucked (trying to depool exec node for T218126) |
[tools.quickstatements] |
| 09:51 |
<akosiaris> |
upgrade kubernetes-node on kubestage1001 (staging cluster) to 1.12.7-1 T220405 |
[production] |
| 09:50 |
<moritzm> |
upgrading snapshot hosts to HHVM 3.18.5+dfsg-1+wmf8+deb9u2 and wikidiff 1.8.1 (T203069) |
[production] |
| 09:40 |
<arturo> |
T218126 hard reboot tools-sgeexec-0918 |
[tools] |
| 09:40 |
<akosiaris> |
upgrade kubernetes-master on neon (staging cluster) to 1.12.7-1 T220405 |
[production] |
| 09:40 |
<akosiaris> |
upgrade kubernetes-master on neon (staging cluster) to 1.12.7-1 |
[production] |
| 09:27 |
<arturo> |
T218126 hard reboot tools-sgeexec-0932 |
[tools] |
| 09:26 |
<arturo> |
T218216 hard reboot tools-sgeexec-0932 |
[tools] |
| 09:05 |
<moritzm> |
upgrading job runners mw1299-mw1311 to HHVM 3.18.5+dfsg-1+wmf8+deb9u2 and wikidiff 1.8.1 (T203069) |
[production] |
| 09:04 |
<arturo> |
T218216 add `profile::ldap::client::labs::client_stack: sssd` to prefix puppet for sge-exec nodes |
[tools] |
| 09:03 |
<arturo> |
T218216 do a controlled rollover of sssd, depooling sgeexec nodes, reboot and repool |
[tools] |
| 08:56 |
<elukey> |
restart druid-broker on druid100[4-6] - stuck after attempt datasource delete action |
[production] |
| 08:46 |
<godog> |
roll-restart swift frontends - T214289 |
[production] |
| 08:39 |
<arturo> |
T218216 disable puppet in all tools-sgeexec-XXXX nodes for controlled sssd rollout |
[tools] |
| 08:36 |
<elukey> |
update thirdparty/cloudera packages to cdh 5.16.1 for jessie/stretch-wikimedia - T218343 |
[production] |
| 08:26 |
<onimisionipe@deploy1001> |
Finished deploy [kartotherian/deploy@f7518bb] (stretch): Insert maps2003 into stretch environment (duration: 00m 22s) |
[production] |
| 08:26 |
<onimisionipe@deploy1001> |
Started deploy [kartotherian/deploy@f7518bb] (stretch): Insert maps2003 into stretch environment |
[production] |
| 08:12 |
<gilles> |
T220265 foreachwiki extensions/WikimediaMaintenance/filebackend/setZoneAccess.php --backend local-multiwrite |
[production] |
| 07:22 |
<mholloway-shell@deploy1001> |
Finished deploy [mobileapps/deploy@efd5bd5]: Revert "Bifurcate imageinfo queries to improve performance" (T220574) (duration: 04m 05s) |
[production] |
| 07:18 |
<mholloway-shell@deploy1001> |
Started deploy [mobileapps/deploy@efd5bd5]: Revert "Bifurcate imageinfo queries to improve performance" (T220574) |
[production] |
| 07:12 |
<onimisionipe> |
depooling maps200[34] to increase cassandra replication factor - T198622 |
[production] |
| 07:09 |
<jijiki> |
Rolling restart thumbor service |
[production] |
| 07:08 |
<jijiki> |
Upgrading Thumbor servers to python-thumbor-wikimedia to 2.4-1+deb9u1 |
[production] |
| 06:59 |
<marostegui> |
Deploy schema change on x1 master, with replication, lag will happen on x1 T217453 |
[production] |