|
2018-08-31
§
|
| 19:12 |
<mutante> |
ms-be2043 - following instructions at https://wikitech.wikimedia.org/wiki/Graphite#Repair_xfs_misreporting_free_space to repair xfs misreporting free space (T199198), fixing docs, icinga-downtime doesn't want fqdn but short name |
[production] |
| 18:10 |
<jforrester@deploy1001> |
Synchronized php-1.32.0-wmf.19/includes/Title.php: Hot-deploy of I05eea553c58 to let users edit [[Copyright]] again (duration: 00m 50s) |
[production] |
| 17:57 |
<mutante> |
depooled wtp2020 because icinga reported memory errors |
[production] |
| 17:39 |
<SMalyshev> |
repooled wdqs1005 |
[production] |
| 17:06 |
<Amir1> |
deploying ec9548f into prod |
[ores] |
| 16:30 |
<jforrester@deploy1001> |
Synchronized php-1.32.0-wmf.19/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.Target.js: Hot-deploy of I38eda4aac48 to fix T203213 (duration: 00m 54s) |
[production] |
| 14:31 |
<elukey> |
re-run webrequest-load-wf-upload-2018-8-31-11, failed due to hadoop workers reboots |
[analytics] |
| 13:33 |
<Amir1> |
spinning up ores-lb-03 to replace ores-lb-02 |
[ores] |
| 13:07 |
<Amir1> |
c3396e3 is going to staging |
[ores] |
| 11:34 |
<hashar> |
cleaned workspace on integration.integration-slave-jessie-1002 |
[releng] |
| 10:49 |
<moritzm> |
installing libgd2 security updates on trusty |
[production] |
| 10:05 |
<elukey> |
re-run webrequest-load-wf-upload-2018-8-31-7, failed due to hadoop workers reboots |
[analytics] |
| 09:46 |
<moritzm> |
installing libx11 security updates on trusty |
[production] |
| 09:15 |
<elukey> |
re-run webrequest-load-wf-upload-2018-8-31-[7,8], failed due to hadoop workers reboots |
[analytics] |
| 08:57 |
<gehel> |
elasticsearch data directory migration on all logstash nodes |
[production] |
| 08:16 |
<godog> |
repair sde1 on ms-be2041 - T199198 |
[production] |
| 07:26 |
<elukey> |
re-run webrequest-load-wf-text-2018-8-31-4, failed due to hadoop workers reboots |
[analytics] |
| 06:41 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/456547 |
[releng] |
| 06:20 |
<elukey> |
re-run webrequest-load-wf-text-2018-8-31-4, failed for hadoop workers reboots |
[analytics] |
| 05:54 |
<elukey> |
resumed the Hadoop workers reboots for kernel upgrades |
[production] |
| 05:33 |
<elukey> |
restart pdfrender on scb1003 |
[production] |
| 04:26 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/456535 |
[releng] |
| 04:09 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/456534 |
[releng] |
| 03:09 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/456531 |
[releng] |
| 02:42 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/456530 |
[releng] |
| 02:28 |
<legoktm> |
deployed https://gerrit.wikimedia.org/r/456528 |
[releng] |
| 00:44 |
<mutante> |
netmon1002 - restarted smokeping, removed radon as target (unblock decome of former dns server), added cobalt instead as a target also in C4 |
[production] |
|
2018-08-30
§
|
| 23:42 |
<mutante> |
vega - removed rsync config and let puppet regenerate it |
[production] |
| 23:38 |
<mutante> |
bromine - remove outdated rsync conf fragment for static-bugzilla, stopping rsync, running puppet |
[production] |
| 23:36 |
<mutante> |
releases1001 - killing outdated rsync processes for releases from bromine |
[production] |
| 22:57 |
<jforrester@deploy1001> |
Finished scap: Full sync for i18n re-build following Ieaded578ffd (duration: 32m 29s) |
[production] |
| 22:25 |
<jforrester@deploy1001> |
Started scap: Full sync for i18n re-build following Ieaded578ffd |
[production] |
| 22:13 |
<volans> |
this was a test for the logging to IRC, please ignore ^^^^ |
[production] |
| 22:12 |
<END> |
(NOTRUN) - Cookbook sre.switchdc.mediawiki.00-disable-puppet (exit_code=0) (switchdc/volans@sarin) |
[production] |
| 22:12 |
<START> |
- Cookbook sre.switchdc.mediawiki.00-disable-puppet (switchdc/volans@sarin) |
[production] |
| 21:57 |
<jforrester@deploy1001> |
Synchronized php-1.32.0-wmf.19/extensions/JsonConfig/: Hot-deploy Ieaded578ffd revert of T200968 due to bugs (duration: 00m 51s) |
[production] |
| 20:47 |
<marxarelli> |
Deploying https://gerrit.wikimedia.org/r/c/integration/config/+/455269 for 18 affected jobs |
[releng] |
| 20:34 |
<Hauskatze> |
github: deleting https://github.com/wikimedia/mediawiki-extensions-SemanticExpressiveness mirror for T199417 |
[releng] |
| 20:31 |
<Hauskatze> |
github: deleting https://github.com/wikimedia/mediawiki-extensions-SemanticLinks mirror for T199422 |
[releng] |
| 20:23 |
<mutante> |
cp1080 - powercycled - lots of RECOVERY from Icinga for IPsec connections - leaving depooled so far (T201174) |
[production] |
| 20:21 |
<Hauskatze> |
github: added standard description to https://github.com/wikimedia/mediawiki-extensions-FundraisingEmailUnsubscribe |
[releng] |
| 20:17 |
<mutante> |
powercycling cp1080 |
[production] |
| 20:06 |
<mutante> |
dzahn@neodymium conftool action : set/pooled=no; selector: name=cp1080.eqiad.wmnet| reason: Strongswan CRITICALs fom Icinga (T201174) |
[production] |
| 20:04 |
<dzahn@neodymium> |
conftool action : set/pooled=no; selector: name=cp1080.eqiad.wmnet |
[production] |
| 19:47 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.32.0-wmf.19 |
[production] |
| 19:38 |
<thcipriani> |
bring integration-slave-docker-1025 back online |
[releng] |
| 19:37 |
<Hauskatze> |
github: deleted mirror of extension-MwEmbedSupport per T197918 |
[releng] |
| 19:36 |
<marxarelli> |
Deploying 1.32.0-wmf.19 to all wikis |
[production] |
| 19:21 |
<krinkle@deploy1001> |
Synchronized php-1.32.0-wmf.19/includes/: I11b390f2e4f5e7 (duration: 01m 16s) |
[production] |
| 19:15 |
<mutante> |
granting project-admin to cwhite |
[puppet] |