1401-1450 of 10000 results (106ms)
2024-08-19 §
08:33 <ayounsi@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox [production]
08:32 <ayounsi@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox [production]
08:32 <ayounsi@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary [production]
08:31 <ayounsi@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary [production]
08:18 <brouberol@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
08:18 <brouberol@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Adding AAAA field to snapshot1010 and dumpsdata1003 - brouberol@cumin1002" [production]
08:18 <brouberol@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Adding AAAA field to snapshot1010 and dumpsdata1003 - brouberol@cumin1002" [production]
08:14 <brouberol@cumin1002> START - Cookbook sre.dns.netbox [production]
07:25 <ayounsi@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox [production]
07:25 <ayounsi@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox [production]
07:25 <ayounsi@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary [production]
07:24 <ayounsi@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary [production]
07:16 <ayounsi@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox [production]
07:16 <ayounsi@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox [production]
07:14 <ayounsi@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary [production]
07:14 <ayounsi@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary [production]
07:00 <marostegui@cumin1002> dbctl commit (dc=all): 'db1195 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P67370 and previous config saved to /var/cache/conftool/dbconfig/20240819-070034-root.json [production]
06:45 <marostegui@cumin1002> dbctl commit (dc=all): 'db1195 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P67369 and previous config saved to /var/cache/conftool/dbconfig/20240819-064528-root.json [production]
06:30 <marostegui@cumin1002> dbctl commit (dc=all): 'db1195 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P67368 and previous config saved to /var/cache/conftool/dbconfig/20240819-063023-root.json [production]
06:15 <marostegui@cumin1002> dbctl commit (dc=all): 'db1195 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P67367 and previous config saved to /var/cache/conftool/dbconfig/20240819-061517-root.json [production]
06:00 <marostegui@cumin1002> dbctl commit (dc=all): 'db1195 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P67366 and previous config saved to /var/cache/conftool/dbconfig/20240819-060011-root.json [production]
05:45 <marostegui@cumin1002> dbctl commit (dc=all): 'db1195 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P67365 and previous config saved to /var/cache/conftool/dbconfig/20240819-054506-root.json [production]
05:30 <marostegui@cumin1002> dbctl commit (dc=all): 'db1195 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P67364 and previous config saved to /var/cache/conftool/dbconfig/20240819-053000-root.json [production]
05:25 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db1195.eqiad.wmnet with reason: Upgrade to 10.6.19 [production]
05:24 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1:00:00 on db1195.eqiad.wmnet with reason: Upgrade to 10.6.19 [production]
05:23 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1195 T372536', diff saved to https://phabricator.wikimedia.org/P67363 and previous config saved to /var/cache/conftool/dbconfig/20240819-052352-root.json [production]
2024-08-18 §
22:03 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Setting back s4 to RW', diff saved to https://phabricator.wikimedia.org/P67362 and previous config saved to /var/cache/conftool/dbconfig/20240818-220355-ladsgroup.json [production]
22:00 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Set s4 as read-only ', diff saved to https://phabricator.wikimedia.org/P67361 and previous config saved to /var/cache/conftool/dbconfig/20240818-220043-ladsgroup.json [production]
20:54 <kamila@cumin2002> dbctl commit (dc=all): 'Setting s4 back to RW', diff saved to https://phabricator.wikimedia.org/P67360 and previous config saved to /var/cache/conftool/dbconfig/20240818-205410-kamila.json [production]
20:50 <kamila@cumin2002> dbctl commit (dc=all): 'Set s4 as read-only due to overload', diff saved to https://phabricator.wikimedia.org/P67359 and previous config saved to /var/cache/conftool/dbconfig/20240818-205024-kamila.json [production]
2024-08-17 §
11:33 <marostegui@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P67358 and previous config saved to /var/cache/conftool/dbconfig/20240817-113358-root.json [production]
11:18 <marostegui@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P67357 and previous config saved to /var/cache/conftool/dbconfig/20240817-111852-root.json [production]
11:03 <marostegui@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P67356 and previous config saved to /var/cache/conftool/dbconfig/20240817-110347-root.json [production]
10:48 <marostegui@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P67355 and previous config saved to /var/cache/conftool/dbconfig/20240817-104841-root.json [production]
10:33 <marostegui@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P67354 and previous config saved to /var/cache/conftool/dbconfig/20240817-103336-root.json [production]
10:18 <marostegui@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P67353 and previous config saved to /var/cache/conftool/dbconfig/20240817-101831-root.json [production]
10:03 <marostegui@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P67352 and previous config saved to /var/cache/conftool/dbconfig/20240817-100325-root.json [production]
09:53 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2154 (T367856)', diff saved to https://phabricator.wikimedia.org/P67351 and previous config saved to /var/cache/conftool/dbconfig/20240817-095320-marostegui.json [production]
09:53 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 7:00:00 on db2154.codfw.wmnet with reason: Maintenance [production]
09:53 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 7:00:00 on db2154.codfw.wmnet with reason: Maintenance [production]
09:52 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 7:00:00 on db2152.codfw.wmnet with reason: Maintenance [production]
09:52 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 7:00:00 on db2152.codfw.wmnet with reason: Maintenance [production]
2024-08-16 §
23:27 <eevans@deploy1003> deploy aborted: Test deploy (duration: 00m 25s) [production]
23:27 <eevans@deploy1003> Started deploy [cassandra/logstash-logback-encoder@42653e6] (beta): Test deploy [production]
20:26 <eevans@deploy1003> Finished deploy [cassandra/logstash-logback-encoder@42653e6] (aqs): Test (duration: 00m 32s) [production]
20:26 <eevans@deploy1003> Started deploy [cassandra/logstash-logback-encoder@42653e6] (aqs): Test [production]
20:15 <eevans@deploy1003> Finished deploy [cassandra/logstash-logback-encoder@42653e6] (beta): Beta deploy (duration: 00m 31s) [production]
20:14 <eevans@deploy1003> Started deploy [cassandra/logstash-logback-encoder@42653e6] (beta): Beta deploy [production]
20:12 <eevans@deploy1003> Finished deploy [restbase/deploy@f696b76] (beta): deploy to beta (duration: 01m 05s) [production]
20:11 <eevans@deploy1003> Started deploy [restbase/deploy@f696b76] (beta): deploy to beta [production]