1201-1250 of 10000 results (63ms)
2022-06-05 §
01:37 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddumps1001.wikimedia.org with OS bullseye [production]
01:37 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
01:22 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
01:08 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddumps1001.wikimedia.org with OS bullseye [production]
01:07 <AntiComposite> deploy 1a0e0eb, 1ddaf1b [tools.stewardbots]
00:19 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
00:18 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddumps1001.wikimedia.org with OS bullseye [production]
2022-06-04 §
23:51 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
23:50 <andrew@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host clouddumps1001.wikimedia.org with OS bullseye [production]
23:32 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
23:29 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddumps1001.wikimedia.org with OS bullseye [production]
18:20 <wm-bot> <multichill> Fixed Flickr bot by casting license[id] to string in /data/project/yifeibot/o/toolserver/bryan/flickr/shared/flickr.py [tools.yifeibot]
17:22 <wm-bot> <multichill> New virtualenv. Pull on pywikibot and other code. First jobs running again on Buster [tools.multichill]
17:18 <wm-bot> <multichill> New virtualenv for Buster. Stuck with stupid venv because of missing mwparserfromhell/wikitextparser [tools.geograph]
17:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1148 (T298560)', diff saved to https://phabricator.wikimedia.org/P29398 and previous config saved to /var/cache/conftool/dbconfig/20220604-170633-ladsgroup.json [production]
17:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1148.eqiad.wmnet with reason: Maintenance [production]
17:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1148.eqiad.wmnet with reason: Maintenance [production]
17:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142 (T298560)', diff saved to https://phabricator.wikimedia.org/P29397 and previous config saved to /var/cache/conftool/dbconfig/20220604-170625-ladsgroup.json [production]
16:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142', diff saved to https://phabricator.wikimedia.org/P29396 and previous config saved to /var/cache/conftool/dbconfig/20220604-165120-ladsgroup.json [production]
16:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142', diff saved to https://phabricator.wikimedia.org/P29395 and previous config saved to /var/cache/conftool/dbconfig/20220604-163615-ladsgroup.json [production]
16:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1135 (T298560)', diff saved to https://phabricator.wikimedia.org/P29394 and previous config saved to /var/cache/conftool/dbconfig/20220604-163340-ladsgroup.json [production]
16:33 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1135.eqiad.wmnet with reason: Maintenance [production]
16:33 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1135.eqiad.wmnet with reason: Maintenance [production]
16:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T298560)', diff saved to https://phabricator.wikimedia.org/P29393 and previous config saved to /var/cache/conftool/dbconfig/20220604-163332-ladsgroup.json [production]
16:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1142 (T298560)', diff saved to https://phabricator.wikimedia.org/P29392 and previous config saved to /var/cache/conftool/dbconfig/20220604-162110-ladsgroup.json [production]
16:18 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P29391 and previous config saved to /var/cache/conftool/dbconfig/20220604-161827-ladsgroup.json [production]
16:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P29390 and previous config saved to /var/cache/conftool/dbconfig/20220604-160321-ladsgroup.json [production]
15:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T298560)', diff saved to https://phabricator.wikimedia.org/P29389 and previous config saved to /var/cache/conftool/dbconfig/20220604-154817-ladsgroup.json [production]
14:00 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
13:58 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddumps1001.wikimedia.org with OS bullseye [production]
13:58 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
13:56 <andrew@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=93) for host clouddumps1001.wikimedia.org with OS bullseye [production]
13:31 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
11:41 <joal> Maunally launch refinery-sqoop-mediawiki-production after manual fix of refinery-sqoop-mediawiki [analytics]
11:39 <joal> Manually sqoop enwiki:user and commonswiki:user and add _SUCCESS flag for following job to kick off [analytics]
07:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1142 (T298560)', diff saved to https://phabricator.wikimedia.org/P29388 and previous config saved to /var/cache/conftool/dbconfig/20220604-072556-ladsgroup.json [production]
07:25 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1142.eqiad.wmnet with reason: Maintenance [production]
07:25 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1142.eqiad.wmnet with reason: Maintenance [production]
05:21 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddumps1001.wikimedia.org with OS bullseye [production]
04:53 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
04:53 <andrew@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=93) for host clouddumps1001.wikimedia.org with OS bullseye [production]
04:28 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
04:24 <andrew@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host clouddumps1001.wikimedia.org with OS bullseye [production]
03:53 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host clouddumps1001.wikimedia.org with OS bullseye [production]
2022-06-03 §
22:19 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
22:19 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
22:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141 (T298560)', diff saved to https://phabricator.wikimedia.org/P29387 and previous config saved to /var/cache/conftool/dbconfig/20220603-221938-ladsgroup.json [production]
22:09 <wm-bot> <root> Shutdown webservice which was in infinite crash loop because of missing $HOME/deno-webservice.bash startup script. Did this ever work? [tools.saami]
22:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1141', diff saved to https://phabricator.wikimedia.org/P29386 and previous config saved to /var/cache/conftool/dbconfig/20220603-220433-ladsgroup.json [production]
22:04 <wm-bot> <root> Stopping webservice due to infinite crash loop caused by bad $HOME/.lighttpd.conf which was last edited in 2014. Assuming this tool has been broken since migration to the stretch grid in 2019. [tools.ramp]