2401-2450 of 10000 results (68ms)
2020-03-20 §
07:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool es1017', diff saved to https://phabricator.wikimedia.org/P10736 and previous config saved to /var/cache/conftool/dbconfig/20200320-073205-marostegui.json [production]
07:26 <marostegui> Restart mysql on es1017 for upgrade - T239791 [production]
07:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool es1017 for update T239791', diff saved to https://phabricator.wikimedia.org/P10735 and previous config saved to /var/cache/conftool/dbconfig/20200320-070945-marostegui.json [production]
07:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Promote es1014 to es3 master, this is a NOOP T239791', diff saved to https://phabricator.wikimedia.org/P10734 and previous config saved to /var/cache/conftool/dbconfig/20200320-070922-marostegui.json [production]
2020-03-19 §
22:15 <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@794f099]: Update mobileapps to 99869f45 (duration: 05m 13s) [production]
22:10 <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@794f099]: Update mobileapps to 99869f45 [production]
19:14 <hashar@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.24 [production]
18:30 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.24/extensions/Wikibase/lib/includes/Store/ByIdDispatchingEntityInfoBuilder.php: [[gerrit:581674|Fix 'max' to Int32EntityId::MAX conversion (T247985)]], part II (duration: 01m 07s) [production]
18:24 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.24/extensions/Wikibase/data-access/src/SingleEntitySourceServices.php: [[gerrit:581674|Fix 'max' to Int32EntityId::MAX conversion (T247985)]], part I (duration: 01m 08s) [production]
17:47 <mutante> releases/releases-jenkins - closed firewall hole to port 80 for caching servers - kept it open just for envoy from the backends - ATS speaks https to them meanwhile [production]
16:54 <hashar@deploy1001> Synchronized php-1.35.0-wmf.24/extensions/RelatedArticles: Do not register "" as a style path, that breaks ResourceLoader - T248090 (duration: 01m 07s) [production]
16:01 <jeh@deploy1001> Finished deploy [horizon/deploy@ad60c2b]: update horizon designate-dashboard submodule (duration: 03m 31s) [production]
15:57 <jeh@deploy1001> Started deploy [horizon/deploy@ad60c2b]: update horizon designate-dashboard submodule [production]
15:19 <andrew@deploy1001> deploy aborted: modest css change for the hiera editing dialog (take two -- I consistently forget to rebase before doing this) (duration: 00m 00s) [production]
14:54 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'production' . [production]
14:52 <otto@deploy1001> helmfile [EQIAD] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
14:48 <otto@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'canary' . [production]
14:48 <otto@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'eventstreams' for release 'production' . [production]
13:32 <hashar@deploy1001> Synchronized php: group1 wikis to 1.35.0-wmf.24 (duration: 01m 07s) [production]
13:31 <hashar@deploy1001> rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.24 [production]
13:11 <marostegui> Rename testwikidatawiki.wb_terms on db1078 - T248086 [production]
12:33 <XioNoX> push frack fw policies T248004 [production]
11:43 <Lucas_WMDE> EU SWAT done [production]
11:40 <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.35.0-wmf.24/includes/OutputPage.php: SWAT: [[gerrit:581245|OutputPage: Fix warning when setting wgUserNewMsgRevisionId (T248049)]] (duration: 01m 08s) [production]
11:15 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: e277d29: trwiki: Grant interface editors editprotected & editsemiprotected (T247672; take II) (duration: 01m 08s) [production]
11:13 <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: e277d29: trwiki: Grant interface editors editprotected & editsemiprotected (T247672) (duration: 01m 07s) [production]
10:47 <ema> upload atskafka 0.4 to buster-wikimedia T237993 [production]
10:24 <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.24/skins/Vector/skin.json: [[gerrit:581248|skins.vector.styles.legacy needs to define legacy feature (T247566)]] (duration: 01m 08s) [production]
10:01 <ema> cp: rolling ats-tls-restart to apply log format changes T248067 T237993 [production]
09:26 <marostegui> m2 maintenance window done T246098 [production]
09:03 <akosiaris> restart gerrit on gerrit1001 T246098 [production]
09:02 <akosiaris> restart otrs-daemon, apache on mendelevium T246098 [production]
09:01 <akosiaris> restart recommendation-api on scb T246098 [production]
09:00 <marostegui> Restart m2 primary database master - T246098 [production]
08:48 <dcausse> depooling wdqs1006 to help catching up lag [production]
08:43 <dcausse> restarting blazegraph on wdqs1006 (T242453) [production]
07:54 <moritzm> installing cups updates from Stretch point release [production]
07:48 <moritzm> installing libjaxen-java security updates from Stretch point release [production]
07:07 <marostegui@deploy1001> Synchronized wmf-config/db-eqiad.php: Update pc1008 spare situation T247787 (duration: 01m 09s) [production]
06:49 <elukey> execute 'sudo rm /etc/logrotate.d/ceph-common' on cloudvirt-dev and cloudcontrol-dev to stop daily cronspam [production]
06:46 <marostegui> Deploy schema change on testcommonswiki.globalimagelinks (empty table) on the s4 master T243987 [production]
06:33 <marostegui> Upgrade db1132 without restarting T246098 [production]
00:39 <twentyafterfour@deploy1001> rebuilt and synchronized wikiversions files: group0 wikiws to 1.35.0-wmf.24 refs T233872 [production]
00:31 <twentyafterfour@deploy1001> Synchronized php-1.35.0-wmf.24/skins/Vector/includes/templates/index.mustache: deploy https://gerrit.wikimedia.org/r/c/mediawiki/skins/Vector/+/581116 which reverts https://gerrit.wikimedia.org/r/c/mediawiki/skins/Vector/+/581054 refs T248010 (duration: 01m 07s) [production]
00:18 <eileen> civicrm revision changed from a1b2cbeac1 to 1c477ff07f, config revision is 37232d8460 [production]
2020-03-18 §
23:31 <twentyafterfour@deploy1001> Synchronized php-1.35.0-wmf.23/includes/TemplateParser.php: sync https://gerrit.wikimedia.org/r/c/mediawiki/core/+/581114/ refs T248010 (duration: 01m 07s) [production]
23:26 <twentyafterfour@deploy1001> Synchronized php-1.35.0-wmf.24/includes/TemplateParser.php: sync https://gerrit.wikimedia.org/r/c/mediawiki/core/+/581115/ (duration: 01m 08s) [production]
22:22 <volans@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
22:18 <volans@cumin1001> START - Cookbook sre.dns.netbox [production]
21:56 <Krinkle> krinkle@mw1385: scap pull # clean up AdHoc debugging for T248010 [production]