6501-6550 of 10000 results (114ms)
2024-10-02 §
01:20 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host cloudlb2004-dev.codfw.wmnet with OS bookworm [production]
01:18 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host logging-hd2005.codfw.wmnet with OS bookworm [production]
2024-10-01 §
23:42 <zabe> zabe@mwmaint2002:~$ cat /home/zabe/s3.txt | xargs -I{} bash -c "echo {}; mwscript extensions/WikimediaMaintenance/migrateESRefToContentTable.php {} --skip /home/zabe/text_table_cleanup/{} --dump /home/zabe/text_table_dump/{} --sleep 1" # T183490 [production]
20:34 <hashar> UTC late backport window completed [production]
20:28 <hashar> mwscript purgeList.php --wiki=tlywiki --namespace=4 # T367009 [production]
20:12 <hashar@deploy2002> Finished scap sync-world: Backport for [[gerrit:1076254|Update wgMetaNamespace for tlywiki (T367009)]] (duration: 07m 21s) [production]
20:07 <hashar@deploy2002> nmw03, hashar: Continuing with sync [production]
20:06 <hashar@deploy2002> nmw03, hashar: Backport for [[gerrit:1076254|Update wgMetaNamespace for tlywiki (T367009)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:04 <hashar@deploy2002> Started scap sync-world: Backport for [[gerrit:1076254|Update wgMetaNamespace for tlywiki (T367009)]] [production]
20:02 <hashar> Restarting CI Jenkins [production]
19:48 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
19:47 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
17:59 <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1077079|Allow storing of passwords for local users in wikitech (T376140)]] (duration: 09m 03s) [production]
17:56 <bking@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
17:55 <ladsgroup@deploy2002> ladsgroup: Continuing with sync [production]
17:55 <bking@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
17:53 <ladsgroup@deploy2002> ladsgroup: Backport for [[gerrit:1077079|Allow storing of passwords for local users in wikitech (T376140)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
17:50 <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1077079|Allow storing of passwords for local users in wikitech (T376140)]] [production]
17:50 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudlb2004-dev.codfw.wmnet with OS bookworm [production]
17:06 <mutante> welcome new deployment user zoe T373666 [production]
17:06 <jhathaway@cumin1002> START - Cookbook sre.hosts.reimage for host sretest1002.eqiad.wmnet with OS bookworm [production]
17:03 <aikochou@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revision-models' for release 'main' . [production]
17:02 <taavi> run extensions/CentralAuth/maintenance/migratePass0 on wikitech [production]
16:41 <ayounsi@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest1002.eqiad.wmnet with OS bookworm [production]
16:32 <aikochou@deploy2002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revision-models' for release 'main' . [production]
16:30 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host cloudlb2004-dev.codfw.wmnet with OS bookworm [production]
16:16 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T364077, test out new flag) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet, repooling both afterwards [production]
16:11 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T364077, test out new flag) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet, repooling both afterwards [production]
16:06 <aikochou@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revision-models' for release 'main' . [production]
16:04 <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1077046|Make Wikitech behave a bit more like a SUL wiki (T371374)]] (duration: 09m 36s) [production]
16:03 <aikochou@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
16:00 <ladsgroup@deploy2002> taavi, ladsgroup: Continuing with sync [production]
15:59 <ryankemper@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T364077, this test transfer should succeed) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet, repooling both afterwards [production]
15:58 <ladsgroup@deploy2002> taavi, ladsgroup: Backport for [[gerrit:1077046|Make Wikitech behave a bit more like a SUL wiki (T371374)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
15:56 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T364077, this test transfer should succeed) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet, repooling both afterwards [production]
15:55 <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1077046|Make Wikitech behave a bit more like a SUL wiki (T371374)]] [production]
15:54 <ryankemper@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T364077, this test transfer should fail) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1023.eqiad.wmnet, repooling both afterwards [production]
15:53 <ryankemper@cumin2002> START - Cookbook sre.wdqs.data-transfer (T364077, this test transfer should fail) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs1023.eqiad.wmnet, repooling both afterwards [production]
15:44 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host logging-hd2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
15:39 <ayounsi@cumin1002> START - Cookbook sre.hosts.reimage for host sretest1002.eqiad.wmnet with OS bookworm [production]
15:36 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host logging-hd2004.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
15:07 <elukey@puppetserver1001> conftool action : set/pooled=yes; selector: name=aux-k8s-ctrl1003.eqiad.wmnet [production]
15:07 <elukey@puppetserver1001> conftool action : set/pooled=yes; selector: name=aux-k8s-worker1003.eqiad.wmnet [production]
15:05 <brennen@deploy2002> Finished deploy [phabricator/deployment@33a2c8d]: deploy phab1004 for T376149 (duration: 01m 07s) [production]
15:04 <brennen@deploy2002> Started deploy [phabricator/deployment@33a2c8d]: deploy phab1004 for T376149 [production]
15:03 <brennen@deploy2002> Finished deploy [phabricator/deployment@33a2c8d]: test deploy phab2002 for T376149 (duration: 00m 30s) [production]
15:03 <jelto@cumin1002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 0:30:00 on phab.wmfusercontent.org with reason: Phabricator/Phorge update [production]
15:03 <jelto@cumin1002> START - Cookbook sre.hosts.downtime for 0:30:00 on phab.wmfusercontent.org with reason: Phabricator/Phorge update [production]
15:03 <brennen@deploy2002> Started deploy [phabricator/deployment@33a2c8d]: test deploy phab2002 for T376149 [production]
15:02 <jelto@cumin1002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 0:30:00 on phab.wmfusercontent.org with reason: Phabricator/Phorge update [production]