1351-1400 of 10000 results (111ms)
2025-03-12 §
08:12 <hashar@deploy2002> Started scap sync-world: Backport for [[gerrit:1125372|Remove obsolete $wgAllowMicrodataAttributes]], [[gerrit:1125374|Remove wgArticlePlaceholderSearchIntegrationBackend (T207407)]], [[gerrit:1125091|Remove obsolete CirrusSearch config]], [[gerrit:1125092|Fix wgCirrusSearchSimilarityProfiles]], [[gerrit:1125097|Remove Cognate legacy settings (T348526)]], [[gerrit:1125124|Remove obsolete $wgFlowMain [production]
08:03 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti1037.eqiad.wmnet [production]
08:02 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2160,2235].codfw.wmnet,db[1176,1217,1228].eqiad.wmnet with reason: m5 master switch T388500 [production]
07:26 <jmm@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ganeti1037.eqiad.wmnet [production]
03:12 <eileen> civicrm upgraded from ec20a105 to 14afd1b8 [production]
2025-03-11 §
22:28 <ejegg> payments-wiki upgraded from 6409fffa to 3d4dfab3 [production]
21:59 <reedy@deploy2002> Synchronized private/: various cleanup (duration: 08m 45s) [production]
20:52 <dzahn@dns1004> END - running authdns-update [production]
20:50 <dzahn@dns1004> START - running authdns-update [production]
20:49 <jhuneidi@deploy2002> Finished scap sync-world: Backport for [[gerrit:1126628|Silence TRX profiler in deferreds after autocreation (T388165)]], [[gerrit:1126630|Silence TRX profiler in deferreds after autocreation (T388165)]] (duration: 13m 05s) [production]
20:42 <jhuneidi@deploy2002> jhuneidi, tgr: Continuing with sync [production]
20:39 <jhuneidi@deploy2002> jhuneidi, tgr: Backport for [[gerrit:1126628|Silence TRX profiler in deferreds after autocreation (T388165)]], [[gerrit:1126630|Silence TRX profiler in deferreds after autocreation (T388165)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:35 <jhuneidi@deploy2002> Started scap sync-world: Backport for [[gerrit:1126628|Silence TRX profiler in deferreds after autocreation (T388165)]], [[gerrit:1126630|Silence TRX profiler in deferreds after autocreation (T388165)]] [production]
20:18 <jhuneidi@deploy2002> Finished scap sync-world: Backport for [[gerrit:1126140|Deploy donate banner to test wiki for event logging testing (T387768)]] (duration: 12m 33s) [production]
20:12 <jhuneidi@deploy2002> ksarabia, jhuneidi: Continuing with sync [production]
20:10 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: cloudelastic1012* for ban host prior to reimage - bking@cumin2002 - T387904 [production]
20:10 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: cloudelastic1012* for ban host prior to reimage - bking@cumin2002 - T387904 [production]
20:09 <jhuneidi@deploy2002> ksarabia, jhuneidi: Backport for [[gerrit:1126140|Deploy donate banner to test wiki for event logging testing (T387768)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:06 <jhuneidi@deploy2002> Started scap sync-world: Backport for [[gerrit:1126140|Deploy donate banner to test wiki for event logging testing (T387768)]] [production]
19:51 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Unbanning all hosts in cloudelastic [production]
19:51 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Unbanning all hosts in cloudelastic [production]
19:37 <jhuneidi@deploy2002> Finished scap sync-world: Backport for [[gerrit:1126634|api: guard against undefined prop relations (T384627)]], [[gerrit:1126633|api: guard against undefined prop relations (T384627)]] (duration: 09m 53s) [production]
19:30 <jhuneidi@deploy2002> reedy, jhuneidi: Continuing with sync [production]
19:30 <jhuneidi@deploy2002> reedy, jhuneidi: Backport for [[gerrit:1126634|api: guard against undefined prop relations (T384627)]], [[gerrit:1126633|api: guard against undefined prop relations (T384627)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
19:27 <jhuneidi@deploy2002> Started scap sync-world: Backport for [[gerrit:1126634|api: guard against undefined prop relations (T384627)]], [[gerrit:1126633|api: guard against undefined prop relations (T384627)]] [production]
19:04 <bking@cumin2002> DONE (PASS) - Cookbook sre.puppet.renew-cert (exit_code=0) for cloudelastic1011.eqiad.wmnet: Renew puppet certificate - bking@cumin2002 [production]
19:00 <bking@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1011.eqiad.wmnet with OS bullseye [production]
18:34 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1011.eqiad.wmnet with reason: host reimage [production]
18:30 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1011.eqiad.wmnet with reason: host reimage [production]
18:25 <jhuneidi@deploy2002> rebuilt and synchronized wikiversions files: group0 to 1.44.0-wmf.20 refs T386215 [production]
18:19 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host cloudelastic1011.eqiad.wmnet with OS bullseye [production]
17:58 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudelastic1011.eqiad.wmnet with OS bullseye [production]
17:48 <swfrench-wmf> mw-(api-ext|web): migrated 50% of residual PHP 7.4 traffic to 8.1 - T383845 [production]
17:46 <swfrench@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-web: apply [production]
17:46 <swfrench@deploy2002> helmfile [codfw] START helmfile.d/services/mw-web: apply [production]
17:46 <swfrench@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-api-ext: apply [production]
17:46 <swfrench@deploy2002> helmfile [codfw] START helmfile.d/services/mw-api-ext: apply [production]
17:43 <swfrench@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-web: apply [production]
17:43 <swfrench@deploy2002> helmfile [codfw] START helmfile.d/services/mw-web: apply [production]
17:43 <swfrench@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-api-ext: apply [production]
17:43 <swfrench@deploy2002> helmfile [codfw] START helmfile.d/services/mw-api-ext: apply [production]
17:39 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudelastic1011.eqiad.wmnet with reason: host reimage [production]
17:39 <swfrench@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-web: apply [production]
17:38 <swfrench@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-web: apply [production]
17:38 <swfrench@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply [production]
17:38 <swfrench@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-api-ext: apply [production]
17:35 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudelastic1011.eqiad.wmnet with reason: host reimage [production]
17:35 <swfrench@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-web: apply [production]
17:35 <swfrench@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-web: apply [production]
17:34 <swfrench@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply [production]