6851-6900 of 10000 results (77ms)
2020-01-30 §
06:23 <vgutierrez> restarting varnish-frontend on cp4030 before it crashes - T243634 [production]
06:21 <vgutierrez> depool cp4032 - T243634 [production]
05:12 <vgutierrez> restarting varnish-frontend and repooling cp4029 - T243634 [production]
05:00 <vgutierrez> depooling cp4029 [production]
2020-01-29 §
23:37 <marostegui> Remove partitions from db2087:3317 - T239453 [production]
18:17 <XioNoX> move knams netflow sampling to cr3-knams [production]
17:19 <krinkle@deploy1001> Synchronized wmf-config/etcd.php: Ice8dad2 (duration: 01m 10s) [production]
01:11 <vgutierrez> varnish-frontend restarted on cp4031 [production]
01:09 <vgutierrez> repool cp4031 [production]
01:05 <marostegui> Disable notifications for dbstore1005:3318 slave lag - T243871 [production]
01:03 <vgutierrez> depool cp4031 [production]
00:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Fully repool db1097:3314 T239453', diff saved to https://phabricator.wikimedia.org/P10289 and previous config saved to /var/cache/conftool/dbconfig/20200129-003507-marostegui.json [production]
00:22 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1097:3314 T239453', diff saved to https://phabricator.wikimedia.org/P10288 and previous config saved to /var/cache/conftool/dbconfig/20200129-002203-marostegui.json [production]
2020-01-28 §
23:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1097:3314 T239453', diff saved to https://phabricator.wikimedia.org/P10287 and previous config saved to /var/cache/conftool/dbconfig/20200128-235336-marostegui.json [production]
23:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1097:3314 T239453', diff saved to https://phabricator.wikimedia.org/P10286 and previous config saved to /var/cache/conftool/dbconfig/20200128-234601-marostegui.json [production]
23:42 <marostegui@cumin1001> dbctl commit (dc=all): 'Start repooling db1084 with its original weight', diff saved to https://phabricator.wikimedia.org/P10285 and previous config saved to /var/cache/conftool/dbconfig/20200128-234219-marostegui.json [production]
23:40 <marostegui@cumin1001> dbctl commit (dc=all): 'Repool db1121 T232446', diff saved to https://phabricator.wikimedia.org/P10284 and previous config saved to /var/cache/conftool/dbconfig/20200128-234037-marostegui.json [production]
15:06 <addshore> Start addshore@mwmaint1002:~$ ./T219123.sh # Taking over from @ladsgroup for T219123 [production]
09:59 <effie> rolling restart mobileapps in codfw [production]
02:05 <mutante> gerrit1002 - gzipping a bunch of /var/log/gerrit/ log files (T243808) [production]
2020-01-27 §
23:40 <eileen> civicrm revision changed from fbd5c35fb0 to ac730a6bcb, config revision is 837b9d0703 [production]
23:10 <vgutierrez> rolling restart of varnish-frontend in cp4026 and cp4027 [production]
23:06 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
23:06 <filippo@cumin1001> START - Cookbook sre.hosts.downtime [production]
23:01 <_joe_> restart apache on gerrit [production]
22:58 <vgutierrez> restarting gerrit service [production]
22:01 <vgutierrez> restarting varnish-fe on cp4028 [production]
19:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db2085:3311 - T239453', diff saved to https://phabricator.wikimedia.org/P10277 and previous config saved to /var/cache/conftool/dbconfig/20200127-191614-marostegui.json [production]
19:15 <marostegui> Remove partitions from db2085 enwiki - T239453 [production]
13:58 <vgutierrez> repooling cp4030 - T243634 [production]
13:54 <vgutierrez> restarting varnish-fe on cp4030 - T243634 [production]
13:54 <vgutierrez> repooling cp4029 - T243634 [production]
13:36 <vgutierrez> restarting varnish-fe on cp4029 - T243634 [production]
12:10 <Amir1> ladsgroup@mwmaint1002:~$ mwscript extensions/Wikibase/repo/maintenance/rebuildItemTerms.php --wiki=wikidatawiki --from-id 1860 --to-id 1860 (T243705) [production]
03:29 <gehel> restarting blazegraph on wdqs100[57] [production]
2020-01-26 §
21:45 <akosiaris> repool maps1003 [production]
21:45 <akosiaris@cumin1001> conftool action : set/pooled=yes; selector: name=maps1003.* [production]
21:42 <akosiaris> test depool maps1003 [production]
21:42 <akosiaris@cumin1001> conftool action : set/pooled=no; selector: name=maps1003.* [production]
21:38 <vgutierrez> powercycling cp3051 - T238305 [production]
21:23 <akosiaris> restart kartotherian on maps1002 [production]
21:19 <vgutierrez> restart varnish-fe and ats-tls on cp3056 [production]
21:02 <bblack> ats-tls-restart on cp3064 [production]
20:51 <bblack> esams text caches: reverting earlier sysctl mitigations [production]
18:11 <volans> shutdown elastic2043 - T243715 [production]
18:01 <volans> depooled elastic2043 - T243715 [production]
18:01 <volans@cumin1001> conftool action : set/pooled=inactive; selector: name=elastic2043.codfw.wmnet [production]
17:28 <elukey> restart varnishkafka-webrequest on cp3064 [production]
17:25 <elukey> restart varnishkafka-webrequest on cp3056 [production]
17:03 <bblack> reduce /proc/sys/net/ipv4/tcp_max_syn_backlog to 8192 on esams text caches [production]