3501-3550 of 10000 results (36ms)
2022-01-03 §
08:28 <ayounsi@cumin1001> START - Cookbook sre.dns.netbox [production]
08:25 <moritzm> installing zziplib security updates [production]
08:07 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2118.codfw.wmnet with reason: Maintenance [production]
08:07 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2118.codfw.wmnet with reason: Maintenance [production]
08:04 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2108.codfw.wmnet with reason: Maintenance [production]
08:04 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2108.codfw.wmnet with reason: Maintenance [production]
07:51 <moritzm> draining primary and secondary instances off ganeti2023 [production]
07:46 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
07:46 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
07:31 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2098.codfw.wmnet with reason: Maintenance [production]
07:31 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2098.codfw.wmnet with reason: Maintenance [production]
07:27 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2087.codfw.wmnet with reason: Maintenance [production]
07:27 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2087.codfw.wmnet with reason: Maintenance [production]
07:23 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2086.codfw.wmnet with reason: Maintenance [production]
07:23 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2086.codfw.wmnet with reason: Maintenance [production]
07:17 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
07:13 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
07:13 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
07:09 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
07:04 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
07:03 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
07:03 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
07:02 <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:750831|Full roll out of wgMaxExecutionTimeForExpensiveQueries (T297708)]], Part I (duration: 01m 20s) [production]
07:01 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
07:00 <ladsgroup@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:750831|Full roll out of wgMaxExecutionTimeForExpensiveQueries (T297708)]], Part I (duration: 00m 58s) [production]
06:53 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db[2077,2095].codfw.wmnet with reason: Maintenance [production]
06:53 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db[2077,2095].codfw.wmnet with reason: Maintenance [production]
04:21 <Amir1> start of running populating actor in revision table on rest of sections. It will take two months to finish (T275246) [production]
2022-01-01 §
23:22 <wm-bot> <lucaswerkmeister> deployed d6110ed631 (l10n updates) [tools.lexeme-forms]
08:19 <taavi> increase quota to 16 cores and 32768 ram T297909 [iiab]
2021-12-31 §
19:48 <taavi> reset grid error status on webgrid-lighttpd@tools-sgewebgrid-lighttpd-0915 [tools]
2021-12-29 §
10:30 <elukey> kill tcpdump process on kubestagemaster1001 (kept a big pcap file opened that kept growing) [production]
2021-12-28 §
20:31 <taavi> restarting acme-chief to debug T298353 [tools]
14:14 <taavi> deploying https://github.com/toolforge/paws/pull/121 [paws]
11:27 <godog> powercycle ms-be1059 -- was powered down [production]
02:39 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
02:35 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
02:35 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
02:31 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
02:21 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
02:17 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
02:17 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn [production]
02:14 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn [production]
2021-12-26 §
16:55 <majavah> run attachLdapUser.php on wikitech for developer account "Karthiksripal" [admin]
2021-12-24 §
22:51 <majavah> ran the wikireplica dns script on s5 T298303 [admin]
21:19 <Amir1> rebooting integration-agent-docker-1009 to pick up new kernel [releng]
20:08 <mforns@deploy1002> Finished deploy [airflow-dags/analytics@e282d2d]: (no justification provided) (duration: 00m 06s) [production]
20:08 <mforns@deploy1002> Started deploy [airflow-dags/analytics@e282d2d]: (no justification provided) [production]
07:58 <majavah> cleared error state from 4 webgrid-lighttpd nodes [tools]
04:20 <legoktm> depooled cp2029 now that it's up [production]