|
2018-08-30
ยง
|
| 22:25 |
<jforrester@deploy1001> |
Started scap: Full sync for i18n re-build following Ieaded578ffd |
[production] |
| 22:13 |
<volans> |
this was a test for the logging to IRC, please ignore ^^^^ |
[production] |
| 22:12 |
<END> |
(NOTRUN) - Cookbook sre.switchdc.mediawiki.00-disable-puppet (exit_code=0) (switchdc/volans@sarin) |
[production] |
| 22:12 |
<START> |
- Cookbook sre.switchdc.mediawiki.00-disable-puppet (switchdc/volans@sarin) |
[production] |
| 21:57 |
<jforrester@deploy1001> |
Synchronized php-1.32.0-wmf.19/extensions/JsonConfig/: Hot-deploy Ieaded578ffd revert of T200968 due to bugs (duration: 00m 51s) |
[production] |
| 20:47 |
<marxarelli> |
Deploying https://gerrit.wikimedia.org/r/c/integration/config/+/455269 for 18 affected jobs |
[releng] |
| 20:34 |
<Hauskatze> |
github: deleting https://github.com/wikimedia/mediawiki-extensions-SemanticExpressiveness mirror for T199417 |
[releng] |
| 20:31 |
<Hauskatze> |
github: deleting https://github.com/wikimedia/mediawiki-extensions-SemanticLinks mirror for T199422 |
[releng] |
| 20:23 |
<mutante> |
cp1080 - powercycled - lots of RECOVERY from Icinga for IPsec connections - leaving depooled so far (T201174) |
[production] |
| 20:21 |
<Hauskatze> |
github: added standard description to https://github.com/wikimedia/mediawiki-extensions-FundraisingEmailUnsubscribe |
[releng] |
| 20:17 |
<mutante> |
powercycling cp1080 |
[production] |
| 20:06 |
<mutante> |
dzahn@neodymium conftool action : set/pooled=no; selector: name=cp1080.eqiad.wmnet| reason: Strongswan CRITICALs fom Icinga (T201174) |
[production] |
| 20:04 |
<dzahn@neodymium> |
conftool action : set/pooled=no; selector: name=cp1080.eqiad.wmnet |
[production] |
| 19:47 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.32.0-wmf.19 |
[production] |
| 19:38 |
<thcipriani> |
bring integration-slave-docker-1025 back online |
[releng] |
| 19:37 |
<Hauskatze> |
github: deleted mirror of extension-MwEmbedSupport per T197918 |
[releng] |
| 19:36 |
<marxarelli> |
Deploying 1.32.0-wmf.19 to all wikis |
[production] |
| 19:21 |
<krinkle@deploy1001> |
Synchronized php-1.32.0-wmf.19/includes/: I11b390f2e4f5e7 (duration: 01m 16s) |
[production] |
| 19:15 |
<mutante> |
granting project-admin to cwhite |
[puppet] |
| 19:14 |
<mutante> |
adding member cwhite |
[puppet] |
| 19:13 |
<krinkle@deploy1001> |
Synchronized php-1.32.0-wmf.19/resources/src/startup: I13a996e01b48 (duration: 01m 06s) |
[production] |
| 18:57 |
<mutante> |
making cwhite a project admin to demo creating instances |
[monitoring] |
| 18:52 |
<moritzm> |
restarting aphlict on phab1001 to pick up nodejs security update |
[production] |
| 18:39 |
<mutante> |
adding new member cwhite |
[monitoring] |
| 16:59 |
<godog> |
xfs_repair on ms-be1041 sdf1 - T199198 (retroactive, started at 15:32 |
[production] |
| 16:35 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=inactive; selector: dc=eqiad,cluster=wdqs,name=wdqs1005.eqiad.wmnet |
[production] |
| 16:30 |
<gehel> |
shutting down wdqs1005 for new SSD and reimaging - T202779 |
[production] |
| 16:29 |
<gehel> |
shutting down wdqs1005 for new SSD and reimaging - T198351 |
[production] |
| 16:15 |
<gehel> |
restart of logstash to move data directory - T198351 |
[production] |
| 16:14 |
<thcipriani> |
reloading zuul to deploy https://gerrit.wikimedia.org/r/#/c/integration/config/+/456362/ |
[releng] |
| 15:59 |
<elukey> |
rerun of pageview-druid-hourly-wf-2018-8-30-13, hadoop worker reboots in progress |
[analytics] |
| 15:46 |
<andrewbogott> |
restarting rabbitmq-server on cloudcontrol1003 |
[admin] |
| 15:23 |
<elukey> |
re-run webrequest-load-wf-upload-2018-8-30-13, failed due to hadoop worker reboots |
[analytics] |
| 15:17 |
<bstorm_> |
increased number of nfsd threads on labstore1004 to 300 |
[production] |
| 14:49 |
<elukey> |
re-run webrequest-load-wf-text-2018-8-30-12, failed due to worker nodes reboots |
[analytics] |
| 14:08 |
<moritzm> |
upgrading mw1262-1265 to wikidiff 1.7.3 |
[production] |
| 13:46 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1119, db1090 (s2 and s7) (duration: 00m 59s) |
[production] |
| 13:41 |
<moritzm> |
upgrading mw1261 to wikidiff 1.7.3 |
[production] |
| 13:31 |
<jynus> |
depooling labsdb1009 due to extra lag |
[production] |
| 13:10 |
<jynus> |
sanitizing fixcopyrightwiki on db2094 and children T202820 |
[production] |
| 13:08 |
<jynus> |
sanitizing fixcopyrightwiki on db1124 and children T202820 |
[production] |
| 13:07 |
<arturo> |
T202636 internal network routing now exists in labtest/labtestn for VM to communicate with each other |
[admin] |
| 13:03 |
<moritzm> |
upgrading grafana to 4.6.4 (security release) |
[production] |
| 12:59 |
<elukey> |
drain + reboot analytics10[28-79]* for kernel updates (will take multiple days) |
[production] |
| 12:28 |
<godog> |
roll restart thumbor in eqiad to upgrade to 2.1 |
[production] |
| 12:25 |
<reedy@deploy1001> |
Synchronized wmf-config/interwiki.php: Updating interwiki cache (duration: 02m 36s) |
[production] |
| 12:23 |
<reedy@deploy1001> |
Synchronized dblists/: fixcopyrightwiki (duration: 00m 56s) |
[production] |
| 12:22 |
<moritzm> |
uploaded grafana 4.6.4 to apt.wikimedia.org |
[production] |
| 12:13 |
<reedy@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: fixcopyrightwiki (duration: 00m 56s) |
[production] |
| 12:11 |
<reedy@deploy1001> |
Synchronized static/images/project-logos: fixcopyrightwiki (duration: 00m 55s) |
[production] |