1901-1950 of 10000 results (66ms)
2020-04-06 ยง
12:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1089 after schema change', diff saved to https://phabricator.wikimedia.org/P10903 and previous config saved to /var/cache/conftool/dbconfig/20200406-125222-marostegui.json [production]
12:46 <urbanecm@deploy1001> rebuilt and synchronized wikiversions files: 77b9ae9: Create grwikimedia [production]
12:44 <urbanecm@deploy1001> Synchronized dblists/: 77b9ae9: Create grwikimedia (duration: 00m 59s) [production]
12:37 <XioNoX> Update eqiad analytics filters with new APT IPs [production]
12:27 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
12:21 <marostegui> Deploy schema change on db1089 [production]
12:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1089 for schema change', diff saved to https://phabricator.wikimedia.org/P10902 and previous config saved to /var/cache/conftool/dbconfig/20200406-122123-marostegui.json [production]
12:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1105:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P10901 and previous config saved to /var/cache/conftool/dbconfig/20200406-122058-marostegui.json [production]
12:14 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
12:08 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
12:04 <godog> test grafana 6.7.2 upgrade on grafana2001 - T244208 [production]
11:57 <awight> EU swat complete [production]
11:53 <awight@deploy1001> Synchronized php-1.35.0-wmf.26/extensions/TwoColConflict: SWAT: [[gerrit:586309|Backport talk page and EventLogging changes (T248243, T249404) (duration: 00m 59s) [production]
11:52 <elukey@cumin1001> END (PASS) - Cookbook sre.aqs.roll-restart (exit_code=0) [production]
11:48 <elukey@cumin1001> START - Cookbook sre.aqs.roll-restart [production]
11:48 <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:586325|Create account creator and rollback groups on yowiki (T249487)]] (duration: 00m 59s) [production]
11:32 <awight@deploy1001> Synchronized php-1.35.0-wmf.26/extensions/ContentTranslation: SWAT: [[gerrit:586311|Avoid failure on restoring draft with no categories (T249400)]] (duration: 01m 02s) [production]
11:25 <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: double-syncing (duration: 00m 58s) [production]
11:24 <marostegui> Deploy schema change on db1105:3311 [production]
11:24 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1105:3311 for schema change', diff saved to https://phabricator.wikimedia.org/P10900 and previous config saved to /var/cache/conftool/dbconfig/20200406-112417-marostegui.json [production]
11:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1099:3311 after schema change', diff saved to https://phabricator.wikimedia.org/P10899 and previous config saved to /var/cache/conftool/dbconfig/20200406-112123-marostegui.json [production]
11:18 <elukey> import AMD ROCm 3.3 packages in buster-wikimedia (component thirdparty/rocm33) - T247082 [production]
11:17 <awight@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:580394|cirrus: Increase commonswiki near match weight (T245642)]] (duration: 00m 59s) [production]
11:11 <awight@deploy1001> Synchronized wmf-config/CommonSettings.php: SWAT: [[gerrit:585779| Whitelist X-Wikimedia-Debug header for cross-wiki API requests (T249107)]] (duration: 00m 59s) [production]
10:51 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:586305| Bumping portals to master (563985)]] (duration: 00m 58s) [production]
10:50 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:586305| Bumping portals to master (563985)]] (duration: 01m 12s) [production]
09:50 <XioNoX> push pfw firewall policies - T249267 [production]
09:40 <marostegui> Deploy schema change on db1099:3311 [production]
09:39 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1099:3311 for schema change', diff saved to https://phabricator.wikimedia.org/P10898 and previous config saved to /var/cache/conftool/dbconfig/20200406-093944-marostegui.json [production]
09:11 <ema> cp2027: upgrade varnish to 5.1.3-1wm13 and restart varnish-fe T249344 [production]
09:08 <ema> upload varnish 5.1.3-1wm13 to buster-wikimedia on apt1001.wm.org T249344 [production]
08:55 <ariel@deploy1001> Finished deploy [dumps/dumps@ae1e705]: add prefetch test, fix multistream index file download link (duration: 00m 09s) [production]
08:55 <ariel@deploy1001> Started deploy [dumps/dumps@ae1e705]: add prefetch test, fix multistream index file download link [production]
08:54 <elukey> bootstrap wdqs200[7,8] - T246343 [production]
08:50 <marostegui> Deploy schema change on db1139:3311 [production]
08:18 <_joe_> conversion of codfw api done [production]
08:07 <marostegui> Deploy schema change on dbstore1003:3311 [production]
07:54 <vgutierrez> rolling restart of ats-tls to disable wmf-analytics log - T249335 T237993 [production]
07:50 <dcausse> search index: deleting stale index wikidatawiki_content_1585224806 on cloudelastic:9243 [production]
07:49 <_joe_> eqiad API migrated to envoy for local TLS termination, now starting codfw [production]
07:35 <elukey> restart elasticsearch_6@cloudelastic-chi-eqiad on cloudelastic1003 as attempt to fix heavy GC runs (old gen) - T231517 [production]
07:35 <marostegui> Rename wb_terms on eqiad excluding labsdb1009, labdb1010, labsdb1011 - T248086 [production]
07:06 <marostegui> Rename wb_terms on codfw - T248086 [production]
06:45 <XioNoX> delete BGP to AS25074 in amsix [production]
06:36 <_joe_> converting the api servers to envoy for TLS in eqiad [production]
06:30 <marostegui> Upgrade dbproxy1019 - T231520 [production]
06:18 <marostegui> Deploy schema change on s1 codfw master, this will generate lag on codfw [production]
05:54 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
05:54 <marostegui@cumin1001> START - Cookbook sre.hosts.decommission [production]
05:50 <vgutierrez> ats-tls restart in cp3056, cp3058 and cp3062 - T249335 [production]