1601-1650 of 10000 results (127ms)
2024-10-30 ยง
13:48 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1084036|Growth [test2wiki]: enable community updates module (T376952)]], [[gerrit:1081099|[Growth] beta: configure the A/B test experiment variants (T377233)]] (duration: 29m 00s) [production]
13:47 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T376905)', diff saved to https://phabricator.wikimedia.org/P70707 and previous config saved to /var/cache/conftool/dbconfig/20241030-134715-ladsgroup.json [production]
13:43 <urbanecm@deploy2002> sgimeno, urbanecm: Continuing with sync [production]
13:32 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P70704 and previous config saved to /var/cache/conftool/dbconfig/20241030-132204-ladsgroup.json [production]
13:20 <moritzm> upgrade PHP 7.4 on mwdebug* to 1:7.4.33-1+0~20221108.73+debian10~1.gbpa00350a+wmf10u2+icu67u3 T378173 [production]
13:19 <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1084036|Growth [test2wiki]: enable community updates module (T376952)]], [[gerrit:1081099|[Growth] beta: configure the A/B test experiment variants (T377233)]] [production]
13:18 <andrewtavis-wmde@deploy2002> Finished deploy [airflow-dags/wmde@ec4746b]: (no justification provided) (duration: 00m 07s) [production]
13:18 <andrewtavis-wmde@deploy2002> Started deploy [airflow-dags/wmde@ec4746b]: (no justification provided) [production]
13:06 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P70703 and previous config saved to /var/cache/conftool/dbconfig/20241030-130657-ladsgroup.json [production]
12:55 <andrewtavis-wmde@deploy2002> Finished deploy [airflow-dags/wmde@ec4746b]: (no justification provided) (duration: 00m 11s) [production]
12:54 <andrewtavis-wmde@deploy2002> Started deploy [airflow-dags/wmde@ec4746b]: (no justification provided) [production]
12:51 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T376905)', diff saved to https://phabricator.wikimedia.org/P70702 and previous config saved to /var/cache/conftool/dbconfig/20241030-125150-ladsgroup.json [production]
12:43 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2156 (T376905)', diff saved to https://phabricator.wikimedia.org/P70701 and previous config saved to /var/cache/conftool/dbconfig/20241030-124316-ladsgroup.json [production]
12:43 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
12:43 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
12:43 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
12:43 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
12:42 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T376905)', diff saved to https://phabricator.wikimedia.org/P70700 and previous config saved to /var/cache/conftool/dbconfig/20241030-124256-ladsgroup.json [production]
12:30 <dreamyjazz@deploy2002> Finished scap sync-world: Backport for [[gerrit:1084743|Handle a missing parent block in GlobalBlockLookup::getUserBlock (T378447)]], [[gerrit:1084742|Handle a missing parent block in GlobalBlockLookup::getUserBlock (T378447)]], [[gerrit:1084748|globalblocks API: Hide autoblocks when target param has username and IP (T377855)]] (duration: 10m 28s) [production]
12:27 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P70699 and previous config saved to /var/cache/conftool/dbconfig/20241030-122749-ladsgroup.json [production]
12:25 <dreamyjazz@deploy2002> dreamyjazz: Continuing with sync [production]
12:22 <mvolz@deploy2002> helmfile [eqiad] DONE helmfile.d/services/citoid: apply [production]
12:22 <dreamyjazz@deploy2002> dreamyjazz: Backport for [[gerrit:1084743|Handle a missing parent block in GlobalBlockLookup::getUserBlock (T378447)]], [[gerrit:1084742|Handle a missing parent block in GlobalBlockLookup::getUserBlock (T378447)]], [[gerrit:1084748|globalblocks API: Hide autoblocks when target param has username and IP (T377855)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
12:22 <mvolz@deploy2002> helmfile [eqiad] START helmfile.d/services/citoid: apply [production]
12:21 <ayounsi@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox [production]
12:21 <ayounsi@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox [production]
12:20 <dreamyjazz@deploy2002> Started scap sync-world: Backport for [[gerrit:1084743|Handle a missing parent block in GlobalBlockLookup::getUserBlock (T378447)]], [[gerrit:1084742|Handle a missing parent block in GlobalBlockLookup::getUserBlock (T378447)]], [[gerrit:1084748|globalblocks API: Hide autoblocks when target param has username and IP (T377855)]] [production]
12:19 <mvolz@deploy2002> helmfile [codfw] DONE helmfile.d/services/citoid: apply [production]
12:19 <mvolz@deploy2002> helmfile [codfw] START helmfile.d/services/citoid: apply [production]
12:18 <ayounsi@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary [production]
12:17 <ayounsi@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary [production]
12:17 <mvolz@deploy2002> helmfile [staging] DONE helmfile.d/services/citoid: apply [production]
12:16 <mvolz@deploy2002> helmfile [staging] START helmfile.d/services/citoid: apply [production]
12:12 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P70698 and previous config saved to /var/cache/conftool/dbconfig/20241030-121242-ladsgroup.json [production]
12:12 <moritzm> installing podman security updates [production]
12:11 <joal@deploy2002> Finished deploy [analytics/refinery@0855ce2] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@0855ce28] (duration: 03m 41s) [production]
12:07 <joal@deploy2002> Started deploy [analytics/refinery@0855ce2] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@0855ce28] [production]
12:04 <joal@deploy2002> Finished deploy [analytics/refinery@0855ce2] (thin): Regular analytics weekly train THIN [analytics/refinery@0855ce28] (duration: 06m 54s) [production]
11:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T376905)', diff saved to https://phabricator.wikimedia.org/P70697 and previous config saved to /var/cache/conftool/dbconfig/20241030-115735-ladsgroup.json [production]
11:57 <joal@deploy2002> Started deploy [analytics/refinery@0855ce2] (thin): Regular analytics weekly train THIN [analytics/refinery@0855ce28] [production]
11:55 <joal@deploy2002> Finished deploy [analytics/refinery@0855ce2]: Regular analytics weekly train [analytics/refinery@0855ce28] (duration: 08m 14s) [production]
11:48 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2149 (T376905)', diff saved to https://phabricator.wikimedia.org/P70696 and previous config saved to /var/cache/conftool/dbconfig/20241030-114808-ladsgroup.json [production]
11:48 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
11:47 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
11:47 <joal@deploy2002> Started deploy [analytics/refinery@0855ce2]: Regular analytics weekly train [analytics/refinery@0855ce28] [production]
11:44 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 12:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
11:43 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 12:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
11:41 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
11:41 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
11:39 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2016.codfw.wmnet [production]