|
2025-04-02
ยง
|
| 23:32 |
<urandom> |
starting `nodetool garbagecollect` on Cassandra/sessionstore1006 |
[production] |
| 23:28 |
<urandom> |
starting `nodetool garbagecollect` on Cassandra/sessionstore2005 |
[production] |
| 22:38 |
<jhathaway> |
puppet private repo changes completed, T385995 |
[production] |
| 22:01 |
<brett> |
Import ncmonitor 1.3.3 into bookworm-wikimedia |
[production] |
| 22:00 |
<dreamyjazz@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1133538|AbuseLogger: properly distinguish between global filters and central DB (T390904)]] (duration: 25m 19s) |
[production] |
| 21:55 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_codfw: test only - bking@cumin2002 - T388610 |
[production] |
| 21:53 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_codfw: test only - bking@cumin2002 - T388610 |
[production] |
| 21:53 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_codfw: test only - bking@cumin2002 - T388610 |
[production] |
| 21:53 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_codfw: test only - bking@cumin2002 - T388610 |
[production] |
| 21:53 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_codfw: test only - bking@cumin2002 - T388610 |
[production] |
| 21:53 |
<dreamyjazz@deploy1003> |
dreamyjazz: Continuing with sync |
[production] |
| 21:53 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_codfw: test only - bking@cumin2002 - T388610 |
[production] |
| 21:53 |
<bking@cumin2002> |
END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_codfw: test only - bking@cumin2002 - T388610 |
[production] |
| 21:52 |
<bking@cumin2002> |
START - Cookbook sre.elasticsearch.rolling-operation Operation.REIMAGE (1 nodes at a time) for ElasticSearch cluster search_codfw: test only - bking@cumin2002 - T388610 |
[production] |
| 21:41 |
<dreamyjazz@deploy1003> |
dreamyjazz: Backport for [[gerrit:1133538|AbuseLogger: properly distinguish between global filters and central DB (T390904)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:37 |
<urandom> |
starting `nodetool garbagecollect` on Cassandra/sessionstore2004 |
[production] |
| 21:35 |
<urandom> |
starting `nodetool garbagecollect` on Cassandra/sessionstore1005 |
[production] |
| 21:35 |
<dreamyjazz@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1133538|AbuseLogger: properly distinguish between global filters and central DB (T390904)]] |
[production] |
| 21:31 |
<reedy@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1133532|Enable EmailAuth enforcement on group 0/1 (T390662)]] (duration: 15m 42s) |
[production] |
| 21:23 |
<reedy@deploy1003> |
reedy, tgr: Continuing with sync |
[production] |
| 21:21 |
<reedy@deploy1003> |
reedy, tgr: Backport for [[gerrit:1133532|Enable EmailAuth enforcement on group 0/1 (T390662)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:15 |
<reedy@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1133532|Enable EmailAuth enforcement on group 0/1 (T390662)]] |
[production] |
| 21:07 |
<reedy@deploy1003> |
Finished scap sync-world: Backport for [[gerrit:1133500|SUL3: Fix user ID mismatch during login (immediately after creation) (T388177)]], [[gerrit:1133502|SUL3: Fix user ID mismatch during login (immediately after creation) (T388177)]], [[gerrit:1133503|Remove redundant WaitConditionLoop from CentralAuthTokenManager]], [[gerrit:1133504|Remove redundant WaitConditionLoop from CentralAuthTokenManager]] |
[production] |
| 21:00 |
<reedy@deploy1003> |
d3r1ck01, matmarex, reedy: Continuing with sync |
[production] |
| 20:56 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1067.eqiad.wmnet' (T381499) |
[admin] |
| 20:52 |
<reedy@deploy1003> |
d3r1ck01, matmarex, reedy: Backport for [[gerrit:1133500|SUL3: Fix user ID mismatch during login (immediately after creation) (T388177)]], [[gerrit:1133502|SUL3: Fix user ID mismatch during login (immediately after creation) (T388177)]], [[gerrit:1133503|Remove redundant WaitConditionLoop from CentralAuthTokenManager]], [[gerrit:1133504|Remove redundant WaitConditionLoop from CentralAuthTokenManager] |
[production] |
| 20:49 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1067.eqiad.wmnet' (T381499) |
[admin] |
| 20:49 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1066.eqiad.wmnet' (T381499) |
[admin] |
| 20:47 |
<reedy@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1133500|SUL3: Fix user ID mismatch during login (immediately after creation) (T388177)]], [[gerrit:1133502|SUL3: Fix user ID mismatch during login (immediately after creation) (T388177)]], [[gerrit:1133503|Remove redundant WaitConditionLoop from CentralAuthTokenManager]], [[gerrit:1133504|Remove redundant WaitConditionLoop from CentralAuthTokenManager]] |
[production] |
| 20:41 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1066.eqiad.wmnet' (T381499) |
[admin] |
| 20:41 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1065.eqiad.wmnet' (T381499) |
[admin] |
| 20:33 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1065.eqiad.wmnet' (T381499) |
[admin] |
| 20:33 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1064.eqiad.wmnet' (T381499) |
[admin] |
| 20:30 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-68, tools-k8s-worker-nfs-55 |
[tools] |
| 20:26 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1064.eqiad.wmnet' (T381499) |
[admin] |
| 20:26 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1063.eqiad.wmnet' (T381499) |
[admin] |
| 20:20 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1063.eqiad.wmnet' (T381499) |
[admin] |
| 20:20 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1062.eqiad.wmnet' (T381499) |
[admin] |
| 20:19 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-68, tools-k8s-worker-nfs-55 |
[tools] |
| 20:14 |
<reedy@deploy1003> |
Started scap sync-world: Backport for [[gerrit:1133500|SUL3: Fix user ID mismatch during login (immediately after creation) (T388177)]], [[gerrit:1133502|SUL3: Fix user ID mismatch during login (immediately after creation) (T388177)]], [[gerrit:1133503|Remove redundant WaitConditionLoop from CentralAuthTokenManager]], [[gerrit:1133504|Remove redundant WaitConditionLoop from CentralAuthTokenManager]] |
[production] |
| 20:12 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1062.eqiad.wmnet' (T381499) |
[admin] |
| 20:12 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1061.eqiad.wmnet' (T381499) |
[admin] |
| 20:06 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1061.eqiad.wmnet' (T381499) |
[admin] |
| 20:06 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1060.eqiad.wmnet' (T381499) |
[admin] |
| 19:59 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1060.eqiad.wmnet' (T381499) |
[admin] |
| 19:59 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1059.eqiad.wmnet' (T381499) |
[admin] |
| 19:57 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudnet1006.eqiad.wmnet' (T381499) |
[admin] |
| 19:54 |
<jhathaway> |
rolling out a change to private repo, 1127150, please let me know if any issues arise when merging patches |
[production] |
| 19:51 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt1059.eqiad.wmnet' (T381499) |
[admin] |
| 19:51 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt1058.eqiad.wmnet' (T381499) |
[admin] |