7601-7650 of 10000 results (46ms)
2024-06-21 §
00:22 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1236', diff saved to https://phabricator.wikimedia.org/P65276 and previous config saved to /var/cache/conftool/dbconfig/20240621-002223-ladsgroup.json [production]
00:08 <mutante> [cp3072:~] $ sudo systemctl start varnishkafka-webrequest.service [production]
00:08 <mutante> [cp3067:~] $ sudo systemctl start logrotate [production]
00:07 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1236 (T352010)', diff saved to https://phabricator.wikimedia.org/P65275 and previous config saved to /var/cache/conftool/dbconfig/20240621-000716-ladsgroup.json [production]
00:00 <sukhe> restarting haproxy on cp3068 and cp3072 [production]
2024-06-20 §
23:47 <zabe@deploy1002> Finished scap: Update interwiki cache (duration: 10m 12s) [production]
23:36 <zabe@deploy1002> Started scap: Update interwiki cache [production]
23:34 <zabe> zabe@mwmaint1002:~$ mwscript extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php --wiki=btmwiki --cluster=all 2>&1 | tee /tmp/btmwiki.UpdateSearchIndexConfig.log # T368038 [production]
23:33 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2161 (T364069)', diff saved to https://phabricator.wikimedia.org/P65274 and previous config saved to /var/cache/conftool/dbconfig/20240620-233346-marostegui.json [production]
23:33 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2161.codfw.wmnet with reason: Maintenance [production]
23:33 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2161.codfw.wmnet with reason: Maintenance [production]
23:33 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T364069)', diff saved to https://phabricator.wikimedia.org/P65273 and previous config saved to /var/cache/conftool/dbconfig/20240620-233324-marostegui.json [production]
23:33 <zabe@deploy1002> Finished scap: Creating btmwiki (T368038) (duration: 12m 20s) [production]
23:20 <zabe@deploy1002> Started scap: Creating btmwiki (T368038) [production]
23:20 <zabe> create Wikipedia Mandailing # T368038 [production]
23:18 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P65272 and previous config saved to /var/cache/conftool/dbconfig/20240620-231817-marostegui.json [production]
23:03 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P65271 and previous config saved to /var/cache/conftool/dbconfig/20240620-230310-marostegui.json [production]
22:48 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T364069)', diff saved to https://phabricator.wikimedia.org/P65270 and previous config saved to /var/cache/conftool/dbconfig/20240620-224803-marostegui.json [production]
22:39 <mutante> aphlict1002/aphlict2001 - systemctl stop aphlict_lograte.timer (and .service); systemctl disable aphlict_logrotate.timer (and .service); systemctl daemon-reload; systemctl reset-failed T367960 [production]
22:33 <zabe@deploy1002> Synchronized wmf-config/InitialiseSettings.php: T361041 T363825 T366649 (duration: 09m 55s) [production]
22:29 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2180 (T367856)', diff saved to https://phabricator.wikimedia.org/P65269 and previous config saved to /var/cache/conftool/dbconfig/20240620-222909-marostegui.json [production]
22:29 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2180.codfw.wmnet with reason: Maintenance [production]
22:28 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2180.codfw.wmnet with reason: Maintenance [production]
22:28 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2169 (T367856)', diff saved to https://phabricator.wikimedia.org/P65268 and previous config saved to /var/cache/conftool/dbconfig/20240620-222847-marostegui.json [production]
22:13 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2169', diff saved to https://phabricator.wikimedia.org/P65267 and previous config saved to /var/cache/conftool/dbconfig/20240620-221340-marostegui.json [production]
21:58 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2169', diff saved to https://phabricator.wikimedia.org/P65266 and previous config saved to /var/cache/conftool/dbconfig/20240620-215833-marostegui.json [production]
21:44 <dancy> Upgraded buildkitd to v0.14.1 in gitlab-cloud-runners staging and production (T367352) [releng]
21:43 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2169 (T367856)', diff saved to https://phabricator.wikimedia.org/P65265 and previous config saved to /var/cache/conftool/dbconfig/20240620-214326-marostegui.json [production]
21:38 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server 14789ac1-bc06-4677-9bb0-66c16c887427 [admin]
21:38 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.migrate_server_to_ovs for server 14789ac1-bc06-4677-9bb0-66c16c887427 [admin]
21:28 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.migrate_dbinstance_to_ovs (exit_code=0) for server tbd [testlabs]
21:23 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.migrate_dbinstance_to_ovs for server tbd [testlabs]
21:23 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.migrate_dbinstance_to_ovs (exit_code=99) for server tbd [testlabs]
21:23 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.migrate_dbinstance_to_ovs for server tbd [testlabs]
21:12 <ebernhardson@deploy1002> helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
21:12 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/page-analytics: apply [production]
21:12 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/page-analytics: apply [production]
21:11 <ebernhardson@deploy1002> helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
21:10 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/media-analytics: apply [production]
21:09 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/media-analytics: apply [production]
21:09 <brett> Include ncmonitor 1.0.0 in wikimedia-bookworm apt repo [production]
21:09 <pfischer@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
21:08 <ebernhardson@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
21:08 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
21:08 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/geo-analytics: apply [production]
21:07 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/geo-analytics: apply [production]
21:07 <ebernhardson@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
21:06 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/editor-analytics: apply [production]
21:06 <swfrench@deploy1002> helmfile [eqiad] START helmfile.d/services/editor-analytics: apply [production]
21:05 <swfrench@deploy1002> helmfile [eqiad] DONE helmfile.d/services/edit-analytics: apply [production]