1-50 of 10000 results (62ms)
2023-02-28 ยง
23:58 <zabe@deploy2002> Started scap: T198673 [production]
23:45 <ejegg> civicrm upgraded from ffc16d2d to d199694e [production]
23:43 <zabe@deploy2002> Synchronized wmf-config/InitialiseSettings.php: T213295 (duration: 06m 56s) [production]
23:24 <mutante> miscweb2002 rm -rf /srv/org/wikimedia/design/blog/ - this has moved to /srv/org/wikimedia/design-blog but was not deleted in codfw - bringing both to the same state before switching design.wikimedia.org over T330090 [production]
23:20 <zabe@deploy2002> Finished scap: Backport for [[gerrit:893066|Drop custom testcommonswiki groups (T213295)]] (duration: 07m 57s) [production]
23:14 <zabe@deploy2002> zabe: Backport for [[gerrit:893066|Drop custom testcommonswiki groups (T213295)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet [production]
23:12 <zabe@deploy2002> Started scap: Backport for [[gerrit:893066|Drop custom testcommonswiki groups (T213295)]] [production]
22:46 <zabe@deploy2002> Synchronized dblists/: close testcommonswiki T213295 (duration: 07m 11s) [production]
22:31 <zabe@deploy2002> Synchronized dblists/: close testcommonswiki T213295 (duration: 06m 40s) [production]
22:24 <brennen@deploy2002> Finished deploy [phabricator/deployment@3f2dd1b]: debug deploy to aphlict2001 (duration: 00m 37s) [production]
22:23 <brennen@deploy2002> Started deploy [phabricator/deployment@3f2dd1b]: debug deploy to aphlict2001 [production]
22:01 <apergos> started rsync from dumpsdata1001 to dumpsdata1004 of /data/otherdumps, running in ariel screen session, no bandwidth cap [production]
22:00 <jclark@cumin1001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
21:57 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
21:50 <bking@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
21:42 <bking@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
21:32 <xcollazo@deploy2002> Finished deploy [airflow-dags/platform_eng@fc4e023]: Deploying section_image_recommendations DAG to platform_eng Airflow instance (duration: 00m 21s) [production]
21:32 <xcollazo@deploy2002> Started deploy [airflow-dags/platform_eng@fc4e023]: Deploying section_image_recommendations DAG to platform_eng Airflow instance [production]
21:26 <urbanecm@deploy2002> Finished scap: Backport for [[gerrit:892529|GrowthExperiments: Enable Growth features by default on testwikis (T330748)]] (duration: 07m 43s) [production]
21:19 <urbanecm@deploy2002> Started scap: Backport for [[gerrit:892529|GrowthExperiments: Enable Growth features by default on testwikis (T330748)]] [production]
21:14 <samtar@deploy2002> Finished scap: Backport for [[gerrit:891836|Disable VectorPromoteAddTopic on production wikis initially (T267444)]] (duration: 10m 36s) [production]
21:05 <samtar@deploy2002> esanders and samtar: Backport for [[gerrit:891836|Disable VectorPromoteAddTopic on production wikis initially (T267444)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet [production]
21:03 <samtar@deploy2002> Started scap: Backport for [[gerrit:891836|Disable VectorPromoteAddTopic on production wikis initially (T267444)]] [production]
20:54 <zabe@deploy2002> Finished scap: Backport for [[gerrit:892967|MessagesGuc: Remove trailing space from NS_TEMPLATE_TALK translation (T330746 T321881)]], [[gerrit:892966|MessagesGuc: Remove trailing space from NS_TEMPLATE_TALK translation (T330746 T321881)]] (duration: 10m 06s) [production]
20:45 <zabe@deploy2002> zabe: Backport for [[gerrit:892967|MessagesGuc: Remove trailing space from NS_TEMPLATE_TALK translation (T330746 T321881)]], [[gerrit:892966|MessagesGuc: Remove trailing space from NS_TEMPLATE_TALK translation (T330746 T321881)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet [production]
20:43 <zabe@deploy2002> Started scap: Backport for [[gerrit:892967|MessagesGuc: Remove trailing space from NS_TEMPLATE_TALK translation (T330746 T321881)]], [[gerrit:892966|MessagesGuc: Remove trailing space from NS_TEMPLATE_TALK translation (T330746 T321881)]] [production]
20:20 <bking@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
20:19 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) [production]
20:16 <bking@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
20:16 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) [production]
20:15 <bking@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
20:14 <bking@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
19:54 <dancy@deploy2002> Started scap: testing [production]
19:51 <dancy@deploy2002> Installing scap version "latest" for 550 hosts [production]
19:28 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['ms-be2073'] [production]
19:24 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ms-be2073'] [production]
19:21 <ryankemper> [WDQS] (Current time) T301167 Re-enabled icinga notifications for `wdqs20[09-12]` [production]
19:21 <ryankemper> [WDQS] (The following was ~20 hours ago, forgot to press enter) T301167 Transferred `/srv/wdqs/categories.jnl` from `wdqs2001` (in-service host) to `wdqs20[09-12]` (new hosts being brought into service) [production]
18:55 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['ms-be2073'] [production]
18:26 <ladsgroup@deploy2002> Finished scap: Backport for [[gerrit:893030|Revert "mwscript: Switch to use run.php"]] (duration: 42m 21s) [production]
18:23 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1418.eqiad.wmnet [production]
18:23 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1417.eqiad.wmnet [production]
18:23 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1416.eqiad.wmnet [production]
18:23 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1415.eqiad.wmnet [production]
18:23 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1448.eqiad.wmnet [production]
18:23 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1447.eqiad.wmnet [production]
18:22 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1450.eqiad.wmnet [production]
18:22 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1414.eqiad.wmnet [production]
18:22 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1449.eqiad.wmnet [production]
18:21 <ladsgroup@cumin1001> conftool action : set/pooled=yes; selector: name=mw1418.eqiad.wmnet [production]