901-950 of 10000 results (60ms)
2019-10-15 §
05:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1098:3316', diff saved to https://phabricator.wikimedia.org/P9338 and previous config saved to /var/cache/conftool/dbconfig/20191015-051236-marostegui.json [production]
05:00 <marostegui@cumin2001> dbctl commit (dc=all): 'Promote db1100 to s5 master and remove read-only from s5 T234300', diff saved to https://phabricator.wikimedia.org/P9337 and previous config saved to /var/cache/conftool/dbconfig/20191015-050042-marostegui.json [production]
05:00 <marostegui@cumin2001> dbctl commit (dc=all): 'Set s5 as read-only for maintenance T234300', diff saved to https://phabricator.wikimedia.org/P9336 and previous config saved to /var/cache/conftool/dbconfig/20191015-050016-marostegui.json [production]
05:00 <marostegui> Starting s5 failover from db1070 to db1100 - T234300 [production]
04:34 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1101:3317 after schema change', diff saved to https://phabricator.wikimedia.org/P9335 and previous config saved to /var/cache/conftool/dbconfig/20191015-043403-marostegui.json [production]
04:15 <marostegui> Start pre-switchover steps T234300 [production]
2019-10-14 §
23:26 <Krinkle> Delete 2019-09-01––2019-09-10 arclamp trace logs from webperf1002, and decompress the rest of 2019-09 (this will trigger svg re-generation), T235425 [production]
23:10 <krinkle@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 86f12b6e (duration: 00m 51s) [production]
21:47 <Krinkle> Deleting 2019-09-01––2019-09-10 arclamp logs on webperf2002, and decompress the rest of 2019-09, T235425 [production]
21:12 <Krinkle> Delete misc arclamp/logs and arclamp/svgs data from between 2018 and and 2019-08 on webperf1002/webperf2002, T235425 [production]
20:41 <maxsem@deploy1001> Synchronized php-1.35.0-wmf.1/includes/: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/542963/ (duration: 00m 55s) [production]
17:56 <mutante> webperf2002 - /srv/xenon/logs/daily# gzip 2019-09*excimer*.log (T235425) [production]
17:21 <onimisionipe@deploy1001> Finished deploy [wdqs/wdqs@217cac5]: New blazegraph build and GUI updates (duration: 16m 45s) [production]
17:04 <onimisionipe@deploy1001> Started deploy [wdqs/wdqs@217cac5]: New blazegraph build and GUI updates [production]
16:07 <moritzm> imported cergen 0.2.4-1+deb10u3 to component/cergen for buster-wikimedia T235405 [production]
16:00 <Urbanecm> Password reset for Xaris333 (T235441) [production]
15:57 <moritzm> imported cergen 0.2.4-1+deb10u2 to component/cergen for buster-wikimedia T235405 [production]
14:28 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1103:3314 for schema change T233625', diff saved to https://phabricator.wikimedia.org/P9329 and previous config saved to /var/cache/conftool/dbconfig/20191014-142843-marostegui.json [production]
14:28 <elukey> upload matomo 3.11 to stretch-wikimedia and upgrade matomo1001 - T234607 [production]
14:21 <marostegui> Deploy schema change on db1116:3317 T234066 T233135 [production]
14:13 <effie> Enable puppet on mw* servers and reload apache - T229792 [production]
13:48 <moritzm> imported cergen 0.2.4-1+deb10u1 to component/cergen for buster-wikimedia T235405 [production]
13:42 <marostegui> Repool labsdb1009 after PSU replacement - T233273 [production]
13:36 <effie> Slowly enable puppet on mw* canaries [production]
13:26 <moritzm> imported python-networkx 1.11-2~wmf1 to component/cergen for buster-wikimedia T235405 [production]
13:21 <@> helmfile [EQIAD] Ran 'apply' command on namespace 'cxserver' for release 'production' . [production]
13:19 <@> helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . [production]
13:18 <@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
13:18 <effie> Disable puppet on mw* to remove php72_only feature flag - T229792 [production]
13:13 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: 245b4e5: Add banwiki logo to IS.php (T234768) (duration: 00m 51s) [production]
13:12 <Urbanecm> Run git reset --hard origin/master in /srv/mediawiki-stagging (deleted https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/542920 and https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/542919 from deployment srv, both don't actually change anything => safe to delete) (T234768) [production]
13:10 <marostegui> Sanitize banwiki on db1124:3313 and db2094:3313 T234770 [production]
12:44 <Amir1> Creating banwiki is banned (done) [production]
12:40 <ladsgroup@deploy1001> Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 03m 04s) [production]
12:34 <ladsgroup@deploy1001> Synchronized langlist: Creating banwiki: T234768 (duration: 00m 50s) [production]
12:32 <ladsgroup@deploy1001> Synchronized static/images/project-logos/: Creating banwiki: T234768 (duration: 00m 51s) [production]
12:31 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Creating banwiki: T234768 (duration: 00m 51s) [production]
12:28 <ladsgroup@deploy1001> rebuilt and synchronized wikiversions files: Creating banwiki: T234768 [production]
12:19 <ladsgroup@deploy1001> Synchronized dblists: Creating banwiki: T234768 (duration: 00m 52s) [production]
12:10 <tarrow@deploy1001> Synchronized php-1.35.0-wmf.1/extensions/Wikibase: SWAT: [[gerrit:542894|Bump up Termbox cache version (T235192)]] (duration: 00m 56s) [production]
11:46 <lucaswerkmeister-wmde@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:514461|Enable reftabs on testwikidata (T199197, T228412)]] (duration: 00m 51s) [production]
11:33 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: a295cc7: Fix wrong domain in wgCopyUploadDomains added in T203363 (T235415) (duration: 00m 51s) [production]
11:27 <kart_> Update cxserver to 2019-10-03-054958-production (T232986) [production]
11:22 <@> helmfile [EQIAD] Ran 'apply' command on namespace 'cxserver' for release 'production' . [production]
11:17 <@> helmfile [CODFW] Ran 'apply' command on namespace 'cxserver' for release 'production' . [production]
11:15 <@> helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . [production]
11:09 <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|538867|Use ContentTranslationEnableMT to disable MT (T232986)]] (duration: 00m 51s) [production]
10:35 <jdrewniak@deploy1001> Synchronized portals: Wikimedia Portals Update: [[gerrit:542893| Bumping portals to master (T128546)]] (duration: 00m 51s) [production]
10:34 <jdrewniak@deploy1001> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:542893| Bumping portals to master (T128546)]] (duration: 00m 52s) [production]
10:07 <marostegui@cumin1001> dbctl commit (dc=all): 'Set db1100 with weight 0 in preparation for tomorrow's failover T234300', diff saved to https://phabricator.wikimedia.org/P9326 and previous config saved to /var/cache/conftool/dbconfig/20191014-100758-marostegui.json [production]