1-50 of 3517 results (11ms)
2022-06-28 §
17:34 <wm-bot2> cleaned up grid queue errors on tools-sgegrid-master (T311538) - cookbook ran by dcaro@vulcanus [tools]
15:51 <taavi> add 4096G cinder quota T311509 [tools]
2022-06-27 §
18:14 <taavi> restart calico, appears to have got stuck after the ca replacement operation [tools]
18:02 <taavi> switchover active cron server to tools-sgecron-2 T284767 [tools]
17:54 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0915.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:52 <wm-bot2> removing grid node tools-sgewebgrid-generic-0902.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:49 <wm-bot2> removing grid node tools-sgeexec-0942.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:15 <taavi> T311412 updating ca used by k8s-apiserver->etcd communication, breakage may happen [tools]
14:58 <taavi> renew puppet ca cert and certificate for tools-puppetmaster-02 T311412 [tools]
14:50 <taavi> backup /var/lib/puppet/server to /root/puppet-ca-backup-2022-06-27.tar.gz on tools-puppetmaster-02 before we do anything else to it T311412 [tools]
2022-06-23 §
17:51 <wm-bot2> removing grid node tools-sgeexec-0941.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:49 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0916.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:46 <wm-bot2> removing grid node tools-sgewebgrid-generic-0901.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:32 <wm-bot2> removing grid node tools-sgeexec-0939.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:30 <wm-bot2> removing grid node tools-sgeexec-0938.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:27 <wm-bot2> removing grid node tools-sgeexec-0937.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:22 <wm-bot2> removing grid node tools-sgeexec-0936.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:19 <wm-bot2> removing grid node tools-sgeexec-0935.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:17 <wm-bot2> removing grid node tools-sgeexec-0934.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:14 <wm-bot2> removing grid node tools-sgeexec-0933.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:11 <wm-bot2> removing grid node tools-sgeexec-0932.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
17:09 <wm-bot2> removing grid node tools-sgeexec-0920.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:30 <wm-bot2> removing grid node tools-sgeexec-0947.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
13:59 <taavi> removing remaining continuous jobs from the stretch grid T277653 [tools]
2022-06-22 §
15:54 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0917.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:51 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0918.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:47 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0919.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:45 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0920.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
2022-06-21 §
15:23 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0914.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:20 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0914.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:18 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0913.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
15:07 <wm-bot2> removing grid node tools-sgewebgrid-lighttpd-0912.tools.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko [tools]
2022-06-03 §
20:07 <wm-bot2> created node tools-sgeweblight-10-26.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by andrew@buster [tools]
19:51 <balloons> Scaling webservice nodes to 20, using new 8G swap flavor T309821 [tools]
19:35 <wm-bot2> created node tools-sgeweblight-10-25.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by andrew@buster [tools]
19:03 <wm-bot2> created node tools-sgeweblight-10-20.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by andrew@buster [tools]
19:01 <wm-bot2> created node tools-sgeweblight-10-19.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by andrew@buster [tools]
18:59 <balloons> depooled old nodes, bringing entirely new grid of nodes online T309821 [tools]
18:22 <wm-bot2> created node tools-sgeweblight-10-17.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by andrew@buster [tools]
17:54 <wm-bot2> created node tools-sgeweblight-10-16.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by andrew@buster [tools]
17:52 <wm-bot2> created node tools-sgeweblight-10-15.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by andrew@buster [tools]
16:59 <andrewbogott> building a bunch of new lighttpd nodes (beginning with tools-sgeweblight-10-12) using a flavor with more swap space [tools]
16:56 <wm-bot2> created node tools-sgeweblight-10-12.tools.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by andrew@buster [tools]
15:50 <balloons> fix fix g3.cores4.ram8.disk20.swap24.ephem20 flavor to include swap. Convert to fix g3.cores4.ram8.disk20.swap8.ephem20 flavor T309821 [tools]
15:50 <balloons> temp add 1.0G swap to sgeweblight hosts T309821 [tools]
15:50 <balloons> fix fix g3.cores4.ram8.disk20.swap24.ephem20 flavor to include swap. Convert to fix g3.cores4.ram8.disk20.swap8.ephem20 flavor t309821 [tools]
15:49 <balloons> temp add 1.0G swap to sgeweblight hosts t309821 [tools]
13:25 <bd808> Upgrading fleet to tools-webservice 0.86 (T309821) [tools]
13:20 <bd808> publish tools-webservice 0.86 (T309821) [tools]
12:46 <taavi> start webservicemonitor on tools-sgecron-01 T309821 [tools]