7351-7400 of 10000 results (54ms)
2021-04-13 §
08:00 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on parse2004.codfw.wmnet with reason: REIMAGE [production]
07:58 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on parse2002.codfw.wmnet with reason: REIMAGE [production]
07:58 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on parse2003.codfw.wmnet with reason: REIMAGE [production]
07:56 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on parse2002.codfw.wmnet with reason: REIMAGE [production]
07:49 <jiji@cumin1001> conftool action : set/pooled=yes; selector: cluster=videoscaler,name=(mw1311.eqiad.wmnet|mw1318.eqiad.wmnet|mw1334.eqiad.wmnet) [production]
07:46 <akosiaris> Start up all components on otrs1001. T279303 [production]
07:38 <jiji@cumin1001> conftool action : set/weight=10; selector: cluster=jobrunner,name=mw1318.eqiad.wmnet [production]
07:38 <jiji@cumin1001> conftool action : set/weight=10; selector: cluster=jobrunner,name=mw1334.eqiad.wmnet [production]
07:30 <akosiaris> migrating to Znuny-6.0.33, release 2021-03-10 . T279303 [production]
07:26 <akosiaris> shutdown all OTRS components on otrs1001, prep for OTRS -> Znuny migration. T279303 [production]
05:56 <_joe_> restarting blazegraph on wdqs1013 [production]
05:44 <eileen> civicrm revision changed from ecc32d2a35 to 76bd8ff009, config revision is c5fc1b91e0 [production]
04:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1112 for schema change', diff saved to https://phabricator.wikimedia.org/P15278 and previous config saved to /var/cache/conftool/dbconfig/20210413-045708-marostegui.json [production]
2021-04-12 §
23:25 <krinkle@deploy1002> Synchronized wmf-config/mc.php: I390b4726d01037107 (duration: 00m 58s) [production]
23:06 <jforrester@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:657696|wgAbuseFilterAflFilterMigrationStage: Make COMPAT_NEW in production (T269712)]] (duration: 00m 58s) [production]
18:31 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 117743f695b5cd4b9fa99ff8aaa00d3f9a1d8889: Enable assignment of importupload on enwikibooks (T278683) (duration: 00m 57s) [production]
18:22 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: a1949fd15a5a6fe745f6b807b2716ccb2a287476: Add extendedconfirmed on svwiki (T279836) (duration: 00m 59s) [production]
18:15 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 5d275ec5378c5faea356aeb0fc985f4d815efed1: Add abusefilter-maintainer to wmgPrivilegedGlobalGroups (T279835) (duration: 00m 58s) [production]
18:10 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 13b10d3b1b0b3ff48077a7d212a0eddd6214ce22: Enable <mapframe> on bswiki (T279635) (duration: 00m 57s) [production]
18:04 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: ae05f7cd53925c06d8a23cb8f667a20d79ce2cff: Replace ombudsman with ombuds in wmgPrivilegedGlobalGroups (T256299) (duration: 00m 57s) [production]
18:03 <urbanecm@deploy1002> sync-file aborted: ae05f7cd53925c06d8a23cb8f667a20d79ce2cff: Replace ombudsman with ombuds in wmgPrivilegedGlobalGroups (T256299ú (duration: 00m 00s) [production]
11:29 <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:678569|Disable legacy javascript in jawiki (T72470)]] (duration: 00m 56s) [production]
11:26 <ladsgroup@deploy1002> Synchronized php-1.36.0-wmf.38/extensions/FlaggedRevs/frontend/FlaggedRevsXML.php: Backport: [[gerrit:678347|Don't do strict equal condition check (T279750)]] (duration: 00m 57s) [production]
11:19 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: NO-OP: 6c03d6a59086fa42ec4fc9d289c819a4d3b8e052: Explicitly set wgGEMentorshipMigrationStage: WRITE_OLD/READ_OLD (T279853) (duration: 00m 58s) [production]
11:13 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:677928|wikidata: post edit constraint jobs on 60% of edits (T204031)]] (duration: 01m 13s) [production]
11:11 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings-labs.php: Config: [[gerrit:677920|Remove all remains of idGeneratorLogging (T274156)]] (2/2, Beta-only) (duration: 00m 56s) [production]
11:10 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:677920|Remove all remains of idGeneratorLogging (T274156)]] (1/2) (duration: 00m 57s) [production]
11:07 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/Wikibase.php: Config: [[gerrit:677560|Remove idGeneratorLogging (T274156)]] (duration: 00m 58s) [production]
11:00 <ladsgroup@deploy1002> Synchronized portals: Wikimedia Portals Update: [[gerrit:678574|Bumping portals to master (T279398 T279419)]] (duration: 00m 58s) [production]
10:59 <ladsgroup@deploy1002> Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:678574|Bumping portals to master (T279398 T279419)]] (duration: 00m 58s) [production]
09:55 <urbanecm@deploy1002> Synchronized private/PrivateSettings.php: Update T250887 mitigations (duration: 00m 57s) [production]
09:44 <Urbanecm> Start server-side upload for 4 video files #2 (T279878, T279839, T279818) [production]
08:43 <Urbanecm> Start server-side upload for 4 video files (T279878, T279839, T279818) [production]
08:08 <jiji@cumin1001> conftool action : set/weight=20; selector: cluster=jobrunner,name=mw1318.eqiad.wmnet [production]
08:07 <jiji@cumin1001> conftool action : set/weight=20; selector: cluster=jobrunner,name=mw1334.eqiad.wmnet [production]
08:07 <jiji@cumin1001> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1311.eqiad.wmnet [production]
08:06 <jiji@cumin1001> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1318.eqiad.wmnet [production]
08:06 <jiji@cumin1001> conftool action : set/pooled=no; selector: cluster=videoscaler,name=mw1334.eqiad.wmnet [production]
08:05 <vgutierrez> restart acme-chief [production]
2021-04-10 §
14:21 <andrew@deploy1002> Finished deploy [horizon/deploy@ee1be56]: fix for T279699 (duration: 04m 12s) [production]
14:17 <andrew@deploy1002> Started deploy [horizon/deploy@ee1be56]: fix for T279699 [production]
14:11 <andrew@deploy1002> Finished deploy [horizon/deploy@ee1be56]: cloudweb2001-dev deploy partial fix for T279699 (duration: 02m 21s) [production]
14:08 <andrew@deploy1002> Started deploy [horizon/deploy@ee1be56]: cloudweb2001-dev deploy partial fix for T279699 [production]
14:08 <andrew@deploy1002> Finished deploy [horizon/deploy@ee1be56]: cloudweb2001-dev deploy partial fix for T279699 (duration: 00m 11s) [production]
14:08 <andrew@deploy1002> Started deploy [horizon/deploy@ee1be56]: cloudweb2001-dev deploy partial fix for T279699 [production]
2021-04-09 §
14:07 <jynus> retry es4 backup dump on eqiad (backup1002) [production]
01:25 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on moss-be2002.codfw.wmnet with reason: REIMAGE [production]
01:23 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime for 2:00:00 on moss-be2002.codfw.wmnet with reason: REIMAGE [production]
00:56 <pt1979@cumin2001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 2:00:00 on moss-be2001.codfw.wmnet with reason: REIMAGE [production]
00:54 <pt1979@cumin2001> START - Cookbook sre.hosts.downtime for 2:00:00 on moss-be2001.codfw.wmnet with reason: REIMAGE [production]