1-50 of 10000 results (29ms)
2019-11-15 ยง
22:14 <jeh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
22:12 <jeh@cumin1001> START - Cookbook sre.hosts.downtime [production]
21:54 <jeh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
21:52 <jeh@cumin1001> START - Cookbook sre.hosts.downtime [production]
21:31 <jeh@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
21:29 <jeh@cumin1001> START - Cookbook sre.hosts.downtime [production]
21:21 <_joe_> disabling proxying to ws on phabricator1003 [production]
20:04 <XioNoX> push pfw policies to pfw3-eqiad - T238368 [production]
20:02 <XioNoX> push pfw policies to pfw3-codfw - T238368 [production]
19:07 <XioNoX> remove vlan 1 trunking between msw1-codfw and mr1-codfw, will cause a quick connectivity issue - T228112 [production]
18:07 <XioNoX> homer push on management switches [production]
17:30 <mutante> phabricator - -started phd service [production]
17:11 <XioNoX> homer push to management routers (https://gerrit.wikimedia.org/r/550576) [production]
16:43 <hashar> Restored zuul-merger / CI for operations/puppet.git [production]
16:29 <hashar> CI slowed down due to a huge spike of internal jobs. Being flushed as of now # T140297 [production]
16:25 <bblack> repool cp2001 [production]
16:08 <bblack> depool cp2001 for experiments [production]
16:02 <moritzm> rebooting rpki1001 to rectify microcode loading [production]
16:00 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
16:00 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:51 <ejegg> updated Fundraising CiviCRM from ae9b3819cd to c05c302e54 [production]
15:36 <ejegg> reduced batch size of CiviCRM contact deduplication jobs [production]
15:11 <ema> pool cp3064 with ATS backend T227432 [production]
15:07 <ema> reboot cp3064 after reimage [production]
14:51 <ema@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:49 <ema@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:25 <ema> depool cp3064 and reimage as text_ats T227432 [production]
14:17 <godog> SIGHUP prometheus@ops on prometheus1004 [production]
14:13 <bblack> lvs1013 - pybal restart for new config [production]
14:13 <bblack> lvs2001 - pybal restart for new config [production]
14:13 <bblack> lvs5001 - pybal restart for new config [production]
14:13 <bblack> lvs4005 - pybal restart for new config [production]
14:12 <bblack> lvs3005 - pybal restart for new config [production]
14:11 <bblack> lvs5003 - pybal restart for new config [production]
14:11 <bblack> lvs4007 - pybal restart for new config [production]
14:11 <bblack> lvs3007 - pybal restart for new config [production]
14:10 <bblack> lvs2004 - pybal restart for new config [production]
14:09 <bblack> lvs1016 - pybal restart for new config [production]
13:28 <ariel@deploy1001> Finished deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts (duration: 00m 03s) [production]
13:28 <ariel@deploy1001> Started deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts [production]
13:06 <ariel@deploy1001> Finished deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts (expecting failure) (duration: 00m 04s) [production]
13:06 <ariel@deploy1001> Started deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts (expecting failure) [production]
11:43 <ariel@deploy1001> Finished deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts (duration: 00m 09s) [production]
11:43 <ariel@deploy1001> Started deploy [dumps/dumps@61090ee]: configuration setting to produce empty abstracts [production]
11:27 <moritzm> reboott ganeti4001-4003 to rectify microcode application [production]
11:26 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:26 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
11:25 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1113:3315 into vslow,dump after schema change', diff saved to https://phabricator.wikimedia.org/P9645 and previous config saved to /var/cache/conftool/dbconfig/20191115-112520-marostegui.json [production]
11:19 <marostegui> Reboot dbproxy2002 [production]
11:15 <marostegui> Reboot dbproxy2004 [production]