4501-4550 of 10000 results (73ms)
2020-04-02 ยง
16:53 <joal@deploy1001> Started deploy [analytics/refinery@5b254c8] (thin): Regular analytics weekly train THIN [analytics/refinery@5b254c8] [production]
16:49 <jforrester@deploy1001> Synchronized php-1.35.0-wmf.26/includes/actions/Action.php: T249162 Partially revert 'WikiPage/Article split. Rely on Article inside Action' (duration: 01m 07s) [production]
16:44 <joal@deploy1001> Finished deploy [analytics/refinery@5b254c8]: Regular analytics weekly train [analytics/refinery@5b254c8] (duration: 13m 50s) [production]
16:37 <volans@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:34 <volans@cumin1001> START - Cookbook sre.dns.netbox [production]
16:34 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Touch and secondary sync of IS for cache-busting (duration: 01m 05s) [production]
16:33 <jforrester@deploy1001> sync-file aborted: T249014 [siwiki] Change wgSitename to drop the ',' (duration: 00m 00s) [production]
16:32 <jforrester@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T249014 [siwiki] Change wgSitename to drop the ',' (duration: 01m 07s) [production]
16:30 <joal@deploy1001> Started deploy [analytics/refinery@5b254c8]: Regular analytics weekly train [analytics/refinery@5b254c8] [production]
16:19 <XioNoX> upgrade netflow4001's fastnetmon to 1.1.4 - T240658 [production]
14:56 <XioNoX> push new test switch config for cloudvirt2001 - T248425 [production]
14:33 <vgutierrez> Enable inbound TLSv1.3 in upload@codfw - T170567 [production]
14:33 <vgutierrez> Enable TLS Session tickets in codfw - T245616 [production]
14:24 <jbond42> updating bluez on ganeti and cloudvirt [production]
14:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1111 after schema change', diff saved to https://phabricator.wikimedia.org/P10865 and previous config saved to /var/cache/conftool/dbconfig/20200402-142338-marostegui.json [production]
14:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1111 after schema change', diff saved to https://phabricator.wikimedia.org/P10864 and previous config saved to /var/cache/conftool/dbconfig/20200402-141802-marostegui.json [production]
14:13 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1111 after schema change', diff saved to https://phabricator.wikimedia.org/P10863 and previous config saved to /var/cache/conftool/dbconfig/20200402-141335-marostegui.json [production]
14:11 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1111 after schema change', diff saved to https://phabricator.wikimedia.org/P10862 and previous config saved to /var/cache/conftool/dbconfig/20200402-141149-marostegui.json [production]
13:50 <marostegui> Compress wbqc_constraints on testcommonswiki and commonswiki (empty tables) - T248967 [production]
13:44 <vgutierrez> update puppet compiler facts [production]
13:40 <marostegui> Deploy schema change on db1111 [production]
13:39 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1111 for schema change', diff saved to https://phabricator.wikimedia.org/P10861 and previous config saved to /var/cache/conftool/dbconfig/20200402-133956-marostegui.json [production]
13:32 <gehel> OSM data reimport on maps2004 - T249086 [production]
12:55 <mutante> mw1390 - mw1399 - pooled and active but status "staged" in netbox, fixing to 'active' [production]
12:52 <mutante> mw1297 - is pooled and serving traffic but status "staged" in netbox. set to "active" [production]
11:40 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1087 after schema change', diff saved to https://phabricator.wikimedia.org/P10858 and previous config saved to /var/cache/conftool/dbconfig/20200402-114020-marostegui.json [production]
10:56 <mutante> decom planet1001 (T248863) [production]
10:56 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
10:55 <dzahn@cumin1001> START - Cookbook sre.hosts.decommission [production]
10:19 <marostegui> Deploy schema change on db1087, this will generate lag on s8 on wiki replicas [production]
10:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1087 for schema change', diff saved to https://phabricator.wikimedia.org/P10857 and previous config saved to /var/cache/conftool/dbconfig/20200402-101920-marostegui.json [production]
10:17 <elukey> set up TLS encryption for all pmacct instances on netflow* to Kafka Jumbo [production]
10:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1104 after schema change', diff saved to https://phabricator.wikimedia.org/P10856 and previous config saved to /var/cache/conftool/dbconfig/20200402-101747-marostegui.json [production]
09:47 <marostegui> Remove haproxy@10.64.37.14 from labsdb hosts - T231280 T248944 [production]
09:40 <gehel> CORRECTION: depool maps2004 for data reimport - T249086 [production]
09:40 <gehel> depool wdqs2004 for data reimport - T249086 [production]
09:33 <oblivian@deploy1001> Finished deploy [docker-pkg/deploy@9f2ba2c]: (no justification provided) (duration: 00m 18s) [production]
09:32 <oblivian@deploy1001> Started deploy [docker-pkg/deploy@9f2ba2c]: (no justification provided) [production]
09:28 <oblivian@deploy1001> Finished deploy [docker-pkg/deploy@4f86d77]: (no justification provided) (duration: 00m 09s) [production]
09:28 <oblivian@deploy1001> Started deploy [docker-pkg/deploy@4f86d77]: (no justification provided) [production]
08:51 <marostegui> Deploy schema change db1104 [production]
08:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1104 for schema change', diff saved to https://phabricator.wikimedia.org/P10854 and previous config saved to /var/cache/conftool/dbconfig/20200402-085057-marostegui.json [production]
08:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1092 after schema change', diff saved to https://phabricator.wikimedia.org/P10853 and previous config saved to /var/cache/conftool/dbconfig/20200402-085019-marostegui.json [production]
08:28 <gehel> repooling wdqs1006 - catched up on lag [production]
08:22 <vgutierrez> Enable inbound TLSv1.3 in upload@esams - T170567 [production]
08:21 <vgutierrez> Enable TLS Session tickets in esams - T245616 [production]
07:45 <moritzm> bounced ferm on ms-be1040 [production]
07:27 <marostegui> Deploy schema change on db1092 [production]
07:27 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1092 for schema change', diff saved to https://phabricator.wikimedia.org/P10850 and previous config saved to /var/cache/conftool/dbconfig/20200402-072730-marostegui.json [production]
07:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1101:3318 after schema change', diff saved to https://phabricator.wikimedia.org/P10849 and previous config saved to /var/cache/conftool/dbconfig/20200402-072500-marostegui.json [production]