3451-3500 of 10000 results (65ms)
2022-05-16 §
11:59 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1081.eqiad.wmnet with reason: T308267 [production]
11:59 <btullis@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1081.eqiad.wmnet with reason: T308267 [production]
11:31 <hnowlan@deploy1002> helmfile [eqiad] DONE helmfile.d/services/tegola-vector-tiles: sync [production]
11:31 <hnowlan@deploy1002> helmfile [eqiad] START helmfile.d/services/tegola-vector-tiles: sync [production]
11:30 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/tegola-vector-tiles: sync [production]
11:30 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: sync [production]
11:26 <XioNoX> asw2-ulsfo fix MTU on 2 interfaces [production]
11:09 <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.10/includes: Backport: [[gerrit:792126|RestrictionStore: Add support for templatelinks migration (T308207)]] (duration: 00m 54s) [production]
11:06 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
11:03 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
11:03 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
11:00 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
10:57 <vgutierrez> test HAProxy 2.4.17 on cp4026 and cp4032 [production]
10:40 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
10:38 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
10:38 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
10:34 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
08:02 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
07:58 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
07:58 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
07:58 <urbanecm> UTC morning B&C window done [production]
07:54 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
07:54 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: e9a00e8: GrowthExperiments: Update campaigns configuration (T305443, T305659, T307521) (duration: 00m 50s) [production]
07:49 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
07:49 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: dc82dfa8: ptwikinews: Enable extension MediaSearch (T299872) (duration: 00m 48s) [production]
07:48 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
07:48 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
07:47 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
07:44 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 57d4a9c: thwikibooks: Enable quiz extension (T308377) (duration: 00m 48s) [production]
07:42 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
07:41 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 3e04f86: thwikibooks: Add more namespaces to wgNamespacesToBeSearchedDefault (T308373) (duration: 00m 48s) [production]
07:39 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
07:39 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
07:38 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
07:36 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 67ce6ce: zhwikisource: Add NS100 to wgNamespacesToBeSearchedDefault (T308393) (duration: 00m 50s) [production]
07:18 <dcausse> restarting blazegraph on wdqs1007 (BlazegraphFreeAllocatorsDecreasingRapidly) [production]
2022-05-15 §
21:47 <aqu@deploy1002> Finished deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) (duration: 00m 07s) [production]
21:46 <aqu@deploy1002> Started deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) [production]
21:42 <aqu@deploy1002> Finished deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) (duration: 00m 07s) [production]
21:42 <aqu@deploy1002> Started deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) [production]
21:39 <aqu@deploy1002> Finished deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) (duration: 00m 08s) [production]
21:39 <aqu@deploy1002> Started deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) [production]
21:30 <aqu@deploy1002> Finished deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) (duration: 00m 08s) [production]
21:30 <aqu@deploy1002> Started deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) [production]
21:14 <aqu@deploy1002> Finished deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) (duration: 00m 08s) [production]
21:14 <aqu@deploy1002> Started deploy [airflow-dags/analytics_test@378e7ca]: (no justification provided) [production]
2022-05-14 §
08:34 <jynus@cumin1001> dbctl commit (dc=all): 'Depool db1172', diff saved to https://phabricator.wikimedia.org/P27830 and previous config saved to /var/cache/conftool/dbconfig/20220514-083421-jynus.json [production]
00:53 <razzi@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on an-tool1005.eqiad.wmnet with reason: Server need to be downgraded to stretch, on monday [production]
00:53 <razzi@cumin1001> START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on an-tool1005.eqiad.wmnet with reason: Server need to be downgraded to stretch, on monday [production]
2022-05-13 §
23:42 <razzi@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on an-tool1007.eqiad.wmnet with reason: Upgrade turnilo [production]