4651-4700 of 10000 results (29ms)
2020-04-29 §
08:31 <vgutierrez> restart ats-tls on cp[3054,3064] [production]
08:07 <moritzm> installing openldap security updates [production]
08:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1105:3311 and 3312 after reimage', diff saved to https://phabricator.wikimedia.org/P11075 and previous config saved to /var/cache/conftool/dbconfig/20200429-080206-marostegui.json [production]
07:55 <marostegui> Upgrade mysql on x1 master (without restarting) in preparation for tomorrow's upgrade - T250701 [production]
07:54 <_joe_> restarting php-fpm on mw1288 (workers die in SIGILL status) [production]
07:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1105:3311 and 3312 after reimage', diff saved to https://phabricator.wikimedia.org/P11074 and previous config saved to /var/cache/conftool/dbconfig/20200429-073144-marostegui.json [production]
07:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1105:3311 and 3312 after reimage', diff saved to https://phabricator.wikimedia.org/P11073 and previous config saved to /var/cache/conftool/dbconfig/20200429-071431-marostegui.json [production]
06:50 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
06:47 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime [production]
06:32 <marostegui> stop mysql on db1105 for reimage [production]
06:22 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1105:3311 and 3312 for reimage', diff saved to https://phabricator.wikimedia.org/P11072 and previous config saved to /var/cache/conftool/dbconfig/20200429-062254-marostegui.json [production]
06:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1103:3314', diff saved to https://phabricator.wikimedia.org/P11071 and previous config saved to /var/cache/conftool/dbconfig/20200429-061941-marostegui.json [production]
06:17 <vgutierrez> ats-tls restart on cp[3050,3058] - T249335 [production]
06:07 <vgutierrez> ats-tls restart on cp3064 - T249335 [production]
05:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1114', diff saved to https://phabricator.wikimedia.org/P11070 and previous config saved to /var/cache/conftool/dbconfig/20200429-054733-marostegui.json [production]
00:56 <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable Growth Study QuickSurvey on enwiki (with sample size 0, for testing) (T248421) (duration: 01m 10s) [production]
00:43 <catrope@deploy1001> Finished scap: Update WikimediaMessages with new i18n messages for T248421 (duration: 55m 23s) [production]
2020-04-28 §
23:48 <catrope@deploy1001> Started scap: Update WikimediaMessages with new i18n messages for T248421 [production]
23:40 <ejegg> updated payments-wiki from 8c896a8247 to afb84cc391 [production]
21:55 <ejegg> updated Payments IPN listener (Standalone SmashPig) from d80e4c5abd to 8c30ed7fe5 [production]
20:57 <cdanis@cumin1001> dbctl commit (dc=all): 's8 weights: -db1111, +db1099,db1101', diff saved to https://phabricator.wikimedia.org/P11069 and previous config saved to /var/cache/conftool/dbconfig/20200428-205739-cdanis.json [production]
18:07 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 0639d9f: Allow bdwikimedia bureaucrats to revoke sysop flag (T251078) (duration: 01m 05s) [production]
18:03 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: 07c28d1: GrowthExperiments: cswiki: Change manual of style to 5 pillars (T251290) (duration: 01m 05s) [production]
17:51 <reedy@deploy1001> Synchronized php-1.35.0-wmf.30/extensions/OAuth: T251306 (duration: 01m 06s) [production]
17:13 <bsitzmann@deploy1001> Finished deploy [mobileapps/deploy@678fb8e]: Update mobileapps to ff88022a (duration: 03m 23s) [production]
17:10 <bsitzmann@deploy1001> Started deploy [mobileapps/deploy@678fb8e]: Update mobileapps to ff88022a [production]
16:37 <jforrester@deploy1001> Synchronized php-1.35.0-wmf.30/includes/Revision/RevisionStore.php: Follow-up If770120: Fix bad combination of type cast and ?? operator (duration: 01m 06s) [production]
16:36 <volker-e@deploy1001> Finished deploy [design/style-guide@335122b]: Deploy design/style-guide: (duration: 00m 08s) [production]
16:36 <volker-e@deploy1001> Started deploy [design/style-guide@335122b]: Deploy design/style-guide: [production]
15:49 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
15:42 <mepps> updated payments-wiki from 45bf1734e0 to 8c896a8247, [production]
15:34 <vgutierrez> rolling restart of ats-tls on cp[3050,3052,3054,3056] - T249335 [production]
15:28 <ppchelko@deploy1001> Finished deploy [changeprop/deploy@2b87a75]: Switch off rules moved to k8s T248677 (duration: 01m 20s) [production]
15:27 <ppchelko@deploy1001> Started deploy [changeprop/deploy@2b87a75]: Switch off rules moved to k8s T248677 [production]
15:15 <hnowlan@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
15:14 <hnowlan@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'changeprop' for release 'production' . [production]
15:13 <otto@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . [production]
15:13 <otto@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . [production]
15:13 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
14:59 <otto@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'eventgate-analytics' for release 'production' . [production]
14:58 <otto@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'production' . [production]
14:58 <otto@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'canary' . [production]
14:56 <otto@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'canary' . [production]
14:55 <otto@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'production' . [production]
14:54 <otto@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'eventgate-main' for release 'production' . [production]
14:54 <otto@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'eventgate-main' for release 'canary' . [production]
14:50 <otto@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'eventgate-analytics' for release 'production' . [production]
14:49 <otto@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'canary' . [production]
14:49 <otto@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'eventgate-logging-external' for release 'production' . [production]
14:45 <otto@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'eventgate-analytics-external' for release 'canary' . [production]