2201-2250 of 10000 results (28ms)
2024-06-27 §
17:33 <swfrench-wmf> canary deployments are healthy, slow-logs still produced, continuing with main deployments for T362978 [production]
12:48 <klausman@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on ml-serve2007.codfw.wmnet with reason: Hardware maintenance for memory errors [production]
12:48 <klausman@cumin2002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on ml-serve2007.codfw.wmnet with reason: Hardware maintenance for memory errors [production]
2024-06-26 §
20:50 <jdrewniak@deploy1002> Finished scap: Backport for [[gerrit:1049972|Enable user pages and select special pages in dark mode (1.43.0-wmf.11) (T366364 T366375 T367375 T367581 T367582 T367583)]] (duration: 08m 09s) [production]
20:44 <jdrewniak@deploy1002> jdlrobson, jdrewniak: Backport for [[gerrit:1049972|Enable user pages and select special pages in dark mode (1.43.0-wmf.11) (T366364 T366375 T367375 T367581 T367582 T367583)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:42 <jdrewniak@deploy1002> Started scap: Backport for [[gerrit:1049972|Enable user pages and select special pages in dark mode (1.43.0-wmf.11) (T366364 T366375 T367375 T367581 T367582 T367583)]] [production]
19:48 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.43.0-wmf.11 refs T366956 [production]
19:02 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.43.0-wmf.11 refs T366956 [production]
18:17 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.43.0-wmf.11 refs T366956 [production]
2024-06-25 §
18:14 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.43.0-wmf.11 refs T366956 [production]
04:01 <mwpresync@deploy1002> Pruned MediaWiki: 1.43.0-wmf.8 (duration: 00m 55s) [production]
03:55 <mwpresync@deploy1002> Finished scap: testwikis wikis to 1.43.0-wmf.11 refs T366956 (duration: 52m 19s) [production]
03:03 <mwpresync@deploy1002> Started scap: testwikis wikis to 1.43.0-wmf.11 refs T366956 [production]
00:01 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on 8 hosts with reason: T365763 [production]
00:01 <brett@cumin2002> START - Cookbook sre.hosts.downtime for 4:00:00 on 8 hosts with reason: T365763 [production]
2024-06-24 §
20:22 <ladsgroup@deploy1002> Synchronized php-1.43.0-wmf.10/includes/libs/rdbms/loadbalancer/LoadBalancer.php: (no justification provided) (duration: 11m 04s) [production]
19:08 <mutante> LDAP - added daphnesmit to group 'wmf' - Phabricator: added dsmit-wmf to WMF-NDA group T368140 [production]
2024-06-21 §
09:31 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-reload (exit_code=0) reloading wikidata_full on wdqs2023.codfw.wmnet from DumpsSource.HDFS (hdfs:///wmf/data/discovery/wikidata/munged_n3_dump/wikidata/full/20240603/ using stat1009.eqiad.wmnet) [production]
2024-06-20 §
22:33 <zabe@deploy1002> Synchronized wmf-config/InitialiseSettings.php: T361041 T363825 T366649 (duration: 09m 55s) [production]
15:04 <eevans@cumin1002> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching sessionstore100[4-6].eqiad.wmnet: Upgrade to Java 11 — T350567 - eevans@cumin1002 [production]
14:46 <eevans@cumin1002> START - Cookbook sre.cassandra.roll-restart for nodes matching sessionstore100[4-6].eqiad.wmnet: Upgrade to Java 11 — T350567 - eevans@cumin1002 [production]
14:25 <elukey@cumin1002> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin1002.eqiad.wmnet with reason: Update wmf-plugin for K8s ml-staging - elukey@cumin1002 [production]
14:13 <elukey@cumin1002> START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin1002.eqiad.wmnet with reason: Update wmf-plugin for K8s ml-staging - elukey@cumin1002 [production]
13:24 <reedy@deploy1002> Synchronized wmf-config/: T368003 (duration: 10m 39s) [production]
08:16 <jnuche@deploy1002> rebuilt and synchronized wikiversions files: group2 wikis to 1.43.0-wmf.10 refs T361404 [production]
2024-06-19 §
16:15 <ayounsi@cumin1002> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) netbox-dev to netbox-dev2003.codfw.wmnet with reason: Netbox 4 on netbox-dev2003 - ayounsi@cumin1002 - T336275 [production]
16:02 <ayounsi@cumin1002> START - Cookbook sre.deploy.python-code netbox-dev to netbox-dev2003.codfw.wmnet with reason: Netbox 4 on netbox-dev2003 - ayounsi@cumin1002 - T336275 [production]
15:55 <ayounsi@cumin1002> END (FAIL) - Cookbook sre.deploy.python-code (exit_code=99) netbox-dev to netbox-dev2003.codfw.wmnet with reason: Netbox 4 on netbox-dev2003 - ayounsi@cumin1002 - T336275 [production]
15:55 <ayounsi@cumin1002> START - Cookbook sre.deploy.python-code netbox-dev to netbox-dev2003.codfw.wmnet with reason: Netbox 4 on netbox-dev2003 - ayounsi@cumin1002 - T336275 [production]
08:18 <jnuche@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.43.0-wmf.10 refs T361404 [production]
2024-06-18 §
18:44 <swfrench-wmf> updated conftool to 3.0.0 on hosts (cp,ncredir) in esams for T365123 [production]
18:38 <swfrench-wmf> updated conftool to 3.0.0 on hosts (cp,ncredir) in eqsin for T365123 [production]
18:33 <swfrench-wmf> updated conftool to 3.0.0 on hosts (cp,ncredir) in drmrs for T365123 [production]
18:27 <swfrench-wmf> updated conftool to 3.0.0 on hosts (cp,ncredir) in magru for T365123 [production]
18:17 <swfrench-wmf> updated conftool to 3.0.0 on hosts (cp,ncredir) in ulsfo for T365123 [production]
17:37 <swfrench-wmf> updated conftool to 3.0.0 on bullseye hosts in eqiad for T365123 [production]
17:35 <swfrench-wmf> updated conftool to 3.0.0 on bookworm hosts in eqiad for T365123 [production]
17:34 <swfrench-wmf> updated conftool to 3.0.0 on buster hosts in eqiad for T365123 [production]
17:16 <swfrench-wmf> updated conftool to 3.0.0 on remaining bullseye hosts in codfw for T365123 [production]
17:14 <swfrench-wmf> updated conftool to 3.0.0 on remaining bookworm hosts in codfw for T365123 [production]
17:12 <swfrench-wmf> updated conftool to 3.0.0 on remaining buster hosts in codfw for T365123 [production]
16:42 <swfrench-wmf> conftool on puppetmaster2001 updated to 3.0.0 for T365123 [production]
16:39 <swfrench-wmf> validated dbctl 3.0.0 on cumin2002 (noop edit to note: on parsercache spare pc2014) for T365123 [production]
16:29 <swfrench-wmf> conftool on cumin2002 updated to 3.0.0 for T365123 [production]
16:23 <swfrench-wmf> depooled / pooled mw2441.codfw.wmnet to smoke-test python3-conftool for T365123 [production]
15:47 <swfrench-wmf> included conftool 3.0.0 into buster/bullseye/bookworm-wikimedia on apt.w.o for T365123 [production]
14:40 <klausman@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on ml-serve2001.codfw.wmnet with reason: Hardware maintenance for memory errors [production]
14:40 <klausman@cumin2002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on ml-serve2001.codfw.wmnet with reason: Hardware maintenance for memory errors [production]
14:09 <swfrench-wmf> included conftool 3.0.0 into buster-wikimedia on apt.w.o for T365123 [production]
08:53 <jnuche@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.43.0-wmf.10 refs T361404 [production]