6451-6500 of 10000 results (111ms)
2023-04-28 §
05:29 <ayounsi@cumin1001> START - Cookbook sre.network.debug for Netbox circuit ID 112 [production]
05:28 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 393731 [production]
05:27 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 393731 [production]
04:08 <eileen> config revision changed from b33fa934 to 2eef4039 [production]
03:16 <ejegg> SmashPig upgraded from db9fa965 to a9fa7a2c [production]
03:08 <ejegg> payments-wiki upgraded from 91582d93 to 61951572 [production]
03:05 <eileen> config revision changed from 98f2afbb to b33fa934 [production]
02:55 <eileen> civicrm upgraded from b4a05476 to e7904ea6 [production]
02:13 <eileen> civicrm upgraded from 601d223e to b4a05476 [production]
2023-04-27 §
22:17 <zabe@deploy1002> Finished scap: T334295 (duration: 06m 58s) [production]
22:10 <zabe@deploy1002> Started scap: T334295 [production]
20:29 <TheresNoTime> close UTC late backport window [production]
20:27 <samtar@deploy1002> Finished scap: Backport for [[gerrit:912884|[cawikisource] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912888|[cawiktionary] Add a wordmark (Vector 2022) (T331823)]] (duration: 07m 19s) [production]
20:21 <samtar@deploy1002> superpes and samtar: Backport for [[gerrit:912884|[cawikisource] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912888|[cawiktionary] Add a wordmark (Vector 2022) (T331823)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet [production]
20:20 <samtar@deploy1002> Started scap: Backport for [[gerrit:912884|[cawikisource] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912888|[cawiktionary] Add a wordmark (Vector 2022) (T331823)]] [production]
20:20 <samtar@deploy1002> Finished scap: Backport for [[gerrit:912874|[cawikibooks] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912877|[cawikinews] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912880|[cawikiquote] Add a wordmark (Vector 2022) (T331823)]] (duration: 09m 43s) [production]
20:11 <samtar@deploy1002> samtar and superpes: Backport for [[gerrit:912874|[cawikibooks] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912877|[cawikinews] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912880|[cawikiquote] Add a wordmark (Vector 2022) (T331823)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet [production]
20:10 <samtar@deploy1002> Started scap: Backport for [[gerrit:912874|[cawikibooks] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912877|[cawikinews] Add a wordmark (Vector 2022) (T331823)]], [[gerrit:912880|[cawikiquote] Add a wordmark (Vector 2022) (T331823)]] [production]
19:27 <xcollazo@deploy1002> Finished deploy [airflow-dags/platform_eng@bc37201]: (no justification provided) (duration: 00m 10s) [production]
19:27 <ejegg> payments-wiki upgraded from 7fa25437 to 91582d93 [production]
19:27 <xcollazo@deploy1002> Started deploy [airflow-dags/platform_eng@bc37201]: (no justification provided) [production]
19:16 <xcollazo@deploy1002> Finished deploy [airflow-dags/platform_eng@f162f4d]: Deploying T333001 on platform_eng Airflow instance. (duration: 12m 01s) [production]
19:03 <xcollazo@deploy1002> Started deploy [airflow-dags/platform_eng@f162f4d]: Deploying T333001 on platform_eng Airflow instance. [production]
18:47 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group2 wikis to 1.41.0-wmf.6 refs T330212 [production]
18:37 <jhuneidi@deploy1002> Finished scap: Backport for [[gerrit:911804|Replace references to actionsToolbar (T335469)]] (duration: 16m 10s) [production]
18:22 <jhuneidi@deploy1002> jhuneidi and jforrester: Backport for [[gerrit:911804|Replace references to actionsToolbar (T335469)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
18:21 <jhuneidi@deploy1002> Started scap: Backport for [[gerrit:911804|Replace references to actionsToolbar (T335469)]] [production]
17:51 <herron@cumin1001> END (FAIL) - Cookbook sre.ganeti.reimage (exit_code=1) for host kafkamon1003.eqiad.wmnet with OS bullseye [production]
17:39 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafkamon1003.eqiad.wmnet with reason: host reimage [production]
17:35 <herron@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on kafkamon1003.eqiad.wmnet with reason: host reimage [production]
17:27 <jnuche@deploy1002> Finished deploy [releng/jenkins-deploy@5a46db1] (releasing): (no justification provided) (duration: 00m 40s) [production]
17:27 <jnuche@deploy1002> Started deploy [releng/jenkins-deploy@5a46db1] (releasing): (no justification provided) [production]
17:14 <hnowlan@deploy1002> Finished deploy [restbase/deploy@a08f56d]: Deploying new wikis: T333272 T334460 T334741 T335020 (duration: 03m 29s) [production]
17:11 <hnowlan@deploy1002> Started deploy [restbase/deploy@a08f56d]: Deploying new wikis: T333272 T334460 T334741 T335020 [production]
17:06 <mutante> deploy2002 - armed the keyholder (sudo keyholder arm and enter passphrase from deployment-key-passphrase in pwstore) - monitoring alert should resolve - T335435 [production]
17:01 <herron@cumin1001> START - Cookbook sre.ganeti.reimage for host kafkamon1003.eqiad.wmnet with OS bullseye [production]
16:56 <volans> uploaded python3-wmflib_1.2.2 to apt.wikimedia.org buster-wikimedia,bullseye-wikimedia,bookworm-wikimedia [production]
16:20 <herron@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host kafkamon1003.eqiad.wmnet [production]
16:20 <herron@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM kafkamon1003.eqiad.wmnet - herron@cumin1001" [production]
16:19 <herron@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM kafkamon1003.eqiad.wmnet - herron@cumin1001" [production]
16:05 <herron@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) kafkamon1003.eqiad.wmnet on all recursors [production]
16:05 <herron@cumin1001> START - Cookbook sre.dns.wipe-cache kafkamon1003.eqiad.wmnet on all recursors [production]
16:05 <herron@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:05 <herron@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM kafkamon1003.eqiad.wmnet - herron@cumin1001" [production]
16:01 <herron@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM kafkamon1003.eqiad.wmnet - herron@cumin1001" [production]
15:59 <herron@cumin1001> START - Cookbook sre.dns.netbox [production]
15:59 <herron@cumin1001> START - Cookbook sre.ganeti.makevm for new host kafkamon1003.eqiad.wmnet [production]
15:58 <vgutierrez> restarting varnish on cp5018 and cp5026 to drop port 80 - T322774 [production]
15:55 <jbond> upload puppetboard_4.3.0-1_all.deb to bookworm-wikimedia [production]
15:37 <legoktm@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-timeline: apply [production]