|
2019-01-25
§
|
| 21:43 |
<krinkle@deploy1001> |
Synchronized wmf-config/PhpAutoPrepend.php: Idb695dd033d42 (duration: 00m 47s) |
[production] |
| 21:05 |
<robh> |
cleared sel on db1068, it had a power redundancy loss event (old and resolved) that was triggering the icinga check |
[production] |
| 20:04 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Pool db1106 as an extra api host (duration: 00m 46s) |
[production] |
| 19:36 |
<jynus> |
powercycle db1114 T214720 |
[production] |
| 19:21 |
<jynus> |
disabling notifications on db1114 |
[production] |
| 19:21 |
<jynus@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1114 (duration: 00m 46s) |
[production] |
| 18:32 |
<bsitzmann@deploy1001> |
Finished deploy [mobileapps/deploy@94b76f5]: Update mobileapps to 4c42e3d (T214714) (duration: 03m 33s) |
[production] |
| 18:28 |
<bsitzmann@deploy1001> |
Started deploy [mobileapps/deploy@94b76f5]: Update mobileapps to 4c42e3d (T214714) |
[production] |
| 17:17 |
<chaomodus> |
notebook1003 restarted nagios-nrpe-server due to oom - T212824 |
[production] |
| 14:43 |
<hashar> |
contint1001: stopping zuul-merger for cleanup duties |
[production] |
| 09:48 |
<marostegui> |
Add dbstore1005:3318 to tendril T210478 |
[production] |
| 08:24 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully repool db1105 (duration: 00m 45s) |
[production] |
| 08:00 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Give more traffic to db1105:3312 (duration: 00m 45s) |
[production] |
| 07:51 |
<elukey> |
restart yarn/hdfs daemons on analytics1056 to pick up new disk settings - T214057 |
[production] |
| 07:40 |
<elukey> |
drain + reboot analytics1054 after disk swap (verify reboot + restore correct fstab mountpoints) - T213038 |
[production] |
| 07:30 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1105:3312 (duration: 00m 45s) |
[production] |
| 07:21 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Slowly repool db1105 (duration: 00m 47s) |
[production] |
| 06:53 |
<marostegui> |
Stop MySQL on db1105 to upgrade MySQL |
[production] |
| 06:53 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Fully depool db1105 (duration: 00m 46s) |
[production] |
| 06:49 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1122 T210713 (duration: 00m 47s) |
[production] |
| 06:13 |
<marostegui> |
Deploy schema change on db1122 - T210713 |
[production] |
| 06:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1122 T210713 (duration: 00m 48s) |
[production] |
| 06:04 |
<marostegui> |
Compress dbstore1002: staging.mep_word_persistence from Aria to InnoDB - T213706 |
[production] |
| 05:42 |
<kartik@deploy1001> |
Finished deploy [cxserver/deploy@a5d7181]: Update cxserver to 356f0a1 (T213257, T213275) (duration: 04m 09s) |
[production] |
| 05:38 |
<kartik@deploy1001> |
Started deploy [cxserver/deploy@a5d7181]: Update cxserver to 356f0a1 (T213257, T213275) |
[production] |
| 03:12 |
<mutante> |
scandium sudo chgrp -R wikidev /srv/deployment/parsoid/deploy/ ; sudo chmod -R g+w /srv/deployment/parsoid/deploy/ (T201366) |
[production] |
| 03:03 |
<mutante> |
scandium - apt-get -t stretch-backports install npm ; run puppet ; remove manually created /apt/preferences.d/npm.pref ; puppet created npm_stretch_backports.pref ; puppet run without errors again (T201366) |
[production] |
| 01:33 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@7770453]: Cleanup deploy - T212524 (duration: 00m 11s) |
[production] |
| 01:33 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@7770453]: Cleanup deploy - T212524 |
[production] |
| 01:28 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@7770453]: Upgrade netbox to 2.5.3 - T212524 Try 2 (duration: 00m 31s) |
[production] |
| 01:27 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@7770453]: Upgrade netbox to 2.5.3 - T212524 Try 2 |
[production] |
| 01:26 |
<crusnov@deploy1001> |
Finished deploy [netbox/deploy@7770453]: Upgrade netbox to 2.5.3 - T212524 (duration: 07m 43s) |
[production] |
| 01:18 |
<crusnov@deploy1001> |
Started deploy [netbox/deploy@7770453]: Upgrade netbox to 2.5.3 - T212524 |
[production] |
| 00:46 |
<ebernhardson@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT T214515 gerrit:486154: Turn on wbsearchentities ab test in de, fr, es (duration: 00m 46s) |
[production] |
| 00:37 |
<ebernhardson@deploy1001> |
Synchronized wmf-config/WikibaseSearchSettings.php: SWAT T214515 gerrit:484334: Add wbsearchentities profiles for de, fr, es (duration: 00m 45s) |
[production] |
| 00:34 |
<ebernhardson@deploy1001> |
Synchronized php-1.33.0-wmf.14/extensions/MobileFrontend/: SWAT T214606 gerrit:486392: MobileFrontend if wikidatadata description exists, set it as tagline (duration: 00m 47s) |
[production] |
| 00:29 |
<ebernhardson@deploy1001> |
Synchronized php-1.33.0-wmf.14/includes/Title.php: SWAT T210739 gerrit:486369: Clone the Title object to prevent mutation (duration: 00m 47s) |
[production] |
| 00:20 |
<ebernhardson@deploy1001> |
Synchronized wmf-config/CirrusSearch-common.php: SWAT T212788 gerrit:485609: autocomplete subphrase matching on wikitech and mw.org 2 of 2 (duration: 00m 45s) |
[production] |
| 00:14 |
<ebernhardson@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT T212788 gerrit:485608: autocomplete subphrase matching on wikitech and mw.org (duration: 00m 46s) |
[production] |
| 00:01 |
<arlolra> |
Updated Parsoid to 4772f44 (T214649, T214648) |
[production] |
|
2019-01-24
§
|
| 23:54 |
<arlolra@deploy1001> |
Finished deploy [parsoid/deploy@f9ef630]: Updating Parsoid to 4772f44 (duration: 11m 58s) |
[production] |
| 23:42 |
<arlolra@deploy1001> |
Started deploy [parsoid/deploy@f9ef630]: Updating Parsoid to 4772f44 |
[production] |
| 22:21 |
<mutante> |
wikitech-static splitting apache2 config files into one file per vhost to make it possible for certbot t odetect them |
[production] |
| 22:11 |
<mutante> |
wikitech-static attempted to use certbot with --authenticator webroot and --installer apache to make it properly work with certbot renew in the future. it created account in /etc/letsencrypt/ made backup in /root/; challenge fails though because all domains need to serve out of a webroot and there is status.wikimedia.org here as well. (T21640) |
[production] |
| 22:08 |
<mutante> |
wikitech-static - certbot was already installed but it wasn't used to generate the existing certs so just running certbot renew did not work, attempted to use certbot to renew but apache plugin missing, installed python-certbot-apache (T214640) |
[production] |
| 21:40 |
<twentyafterfour> |
Finished MediaWiki train for 1.33.0-wmf.14 (T206668) - there is no train next week so I'll be back with wmf.16 (T206670) in two weeks. |
[production] |
| 21:16 |
<twentyafterfour@deploy1001> |
Synchronized php-1.33.0-wmf.14/extensions/AbuseFilter/includes/Views/AbuseFilterView.php: sync I67ca475fa16ea449820f8c735531c2cc1b0ec975 refs T206668 (duration: 00m 47s) |
[production] |
| 20:47 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.14 refs T206668 |
[production] |
| 20:10 |
<jforrester@deploy1001> |
Finished scap: Post-SWAT full sync for new i18n for T208097 (duration: 33m 54s) |
[production] |
| 19:59 |
<mutante> |
temp disabled puppet on phab1001 , applying ferm change to allow deployment servers to http to phab servers |
[production] |