6301-6350 of 10000 results (82ms)
2022-12-14 §
09:53 <hnowlan@puppetmaster1001> conftool action : set/weight=2:pooled=yes; selector: service=thumbor,name=kubernetes1010.eqiad.wmnet [production]
09:47 <elukey@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-cache1003.eqiad.wmnet [production]
09:46 <ladsgroup@deploy1002> Synchronized php-1.40.0-wmf.14/includes/search/SearchResultThumbnailProvider.php: Backport: [[gerrit:868046|search: Avoid setting height in search thumbnails (T322621)]] (duration: 08m 07s) [production]
09:46 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast4003.wikimedia.org [production]
09:45 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-cache1002.eqiad.wmnet [production]
09:41 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-corp1001.wikimedia.org [production]
09:40 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host bast4003.wikimedia.org [production]
09:39 <elukey@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-cache1002.eqiad.wmnet [production]
09:39 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast5002.wikimedia.org [production]
09:37 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-corp1001.wikimedia.org [production]
09:35 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-corp2001.wikimedia.org [production]
09:34 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ml-cache1001.eqiad.wmnet [production]
09:30 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-corp2001.wikimedia.org [production]
09:29 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host bast5002.wikimedia.org [production]
09:28 <jayme@deploy1002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'apply'. [production]
09:27 <jayme@deploy1002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'apply'. [production]
09:27 <jayme@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
09:27 <jayme@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
09:27 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast6001.wikimedia.org [production]
09:26 <elukey@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-cache1001.eqiad.wmnet [production]
09:25 <jayme@deploy1002> helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
09:21 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host bast6001.wikimedia.org [production]
09:20 <jayme@deploy1002> helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
08:41 <hashar> Restarted Gerrit for a plugin update [production]
08:39 <hashar@deploy1002> Finished deploy [gerrit/gerrit@c0b0a70]: Add support for PipelineBot to the Checks API plugin - T214068 (duration: 00m 09s) [production]
08:39 <hashar@deploy1002> Started deploy [gerrit/gerrit@c0b0a70]: Add support for PipelineBot to the Checks API plugin - T214068 [production]
08:39 <aqu@deploy1002> Finished deploy [airflow-dags/analytics@353573b]: HDFS usage dataset pipeline deployment without superuser [airflow-dags@353573b] (duration: 00m 13s) [production]
08:39 <aqu@deploy1002> Started deploy [airflow-dags/analytics@353573b]: HDFS usage dataset pipeline deployment without superuser [airflow-dags@353573b] [production]
08:37 <aqu@deploy1002> Finished deploy [airflow-dags/analytics_test@353573b]: HDFS usage dataset pipeline deployment without superuser TEST [airflow-dags@353573b] (duration: 00m 10s) [production]
08:37 <aqu@deploy1002> Started deploy [airflow-dags/analytics_test@353573b]: HDFS usage dataset pipeline deployment without superuser TEST [airflow-dags@353573b] [production]
08:26 <hashar@deploy1002> Finished deploy [gerrit/gerrit@c0b0a70]: Add support for PipelineBot to the Checks API plugin - T214068 (duration: 00m 11s) [production]
08:25 <hashar@deploy1002> Started deploy [gerrit/gerrit@c0b0a70]: Add support for PipelineBot to the Checks API plugin - T214068 [production]
07:28 <phedenskog@deploy1002> Finished deploy [performance/navtiming@7ba179f]: (no justification provided) (duration: 00m 08s) [production]
07:28 <phedenskog@deploy1002> Started deploy [performance/navtiming@7ba179f]: (no justification provided) [production]
01:22 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host logstash2037.codfw.wmnet with OS bullseye [production]
01:18 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host logstash2036.codfw.wmnet with OS bullseye [production]
01:06 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on logstash2037.codfw.wmnet with reason: host reimage [production]
01:04 <cwhite@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on logstash2036.codfw.wmnet with reason: host reimage [production]
01:02 <cwhite@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on logstash2037.codfw.wmnet with reason: host reimage [production]
01:01 <cwhite@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on logstash2036.codfw.wmnet with reason: host reimage [production]
00:46 <cwhite@cumin2002> START - Cookbook sre.hosts.reimage for host logstash2037.codfw.wmnet with OS bullseye [production]
00:45 <cwhite@cumin2002> START - Cookbook sre.hosts.reimage for host logstash2036.codfw.wmnet with OS bullseye [production]
2022-12-13 §
22:57 <jdrewniak@deploy1002> Finished scap: Backport for [[gerrit:867618|Account for syntax errors in closest selector (T325113)]] (duration: 08m 29s) [production]
22:50 <jdrewniak@deploy1002> jdrewniak and jdlrobson: Backport for [[gerrit:867618|Account for syntax errors in closest selector (T325113)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
22:48 <jdrewniak@deploy1002> Started scap: Backport for [[gerrit:867618|Account for syntax errors in closest selector (T325113)]] [production]
22:42 <jdrewniak@deploy1002> Finished scap: Backport for [[gerrit:867617|Account for syntax errors in closest selector (T325113)]] (duration: 09m 20s) [production]
22:40 <dzahn@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "parse1002: failed -> active - dzahn@cumin2002" [production]
22:40 <mutante> netbox: set parse1002 status: failed -> active in web UI; ran cookbook 'sre.puppet.sync-netbox-hiera' to get data in sync - T324949 [production]
22:38 <dzahn@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "parse1002: failed -> active - dzahn@cumin2002" [production]
22:35 <dzahn@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]