|
2017-04-10
§
|
| 10:23 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=maps1002.eqiad.wmnet |
[production] |
| 10:12 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=maps1002.eqiad.wmnet |
[production] |
| 10:11 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=maps1001.eqiad.wmnet |
[production] |
| 10:03 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=maps1001.eqiad.wmnet |
[production] |
| 10:02 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=maps2004.codfw.wmnet |
[production] |
| 10:01 |
<gehel> |
rolling restart of maps1* (eqiad) cluster |
[production] |
| 09:52 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=maps2004.codfw.wmnet |
[production] |
| 09:52 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=maps2003.codfw.wmnet |
[production] |
| 09:44 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=maps2003.codfw.wmnet |
[production] |
| 09:44 |
<XioNoX> |
all interfaces back up on cr2-esams, BGP sessions up as well T162239 |
[production] |
| 09:44 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=maps2002.codfw.wmnet |
[production] |
| 09:33 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=maps2002.codfw.wmnet |
[production] |
| 09:29 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=maps2001.codfw.wmnet |
[production] |
| 09:18 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=maps2001.codfw.wmnet |
[production] |
| 09:17 |
<XioNoX> |
remote hands work started to replace the FPC on cr2-esams T162239 |
[production] |
| 09:16 |
<gehel> |
rolling restart of maps2* cluster |
[production] |
| 08:52 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: dc=codfw,cluster=wdqs |
[production] |
| 08:51 |
<godog> |
swift codfw-prod: bump ms-be2028 ms-be2039 object weight to 4000 - T158337 |
[production] |
| 08:48 |
<gehel> |
reimage elastic2020 - T149006 |
[production] |
| 08:43 |
<gehel> |
rolling restart of maps-test cluster |
[production] |
| 08:39 |
<elukey> |
manual failover of Hadoop master daemons from analyitics1001 to analytics1002 (T160333) |
[production] |
| 07:48 |
<_joe_> |
testing a dry-run of the switchdc software on sarin |
[production] |
| 07:02 |
<moritzm> |
installing pam updates from jessie point update |
[production] |
| 06:26 |
<marostegui> |
Deploy schema change labsdb1001 (s7) - T160390 |
[production] |
| 06:24 |
<marostegui> |
Deploy schema change db1028 (s7) - T160390 |
[production] |
| 06:24 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1028 - T160390 (duration: 00m 39s) |
[production] |
| 06:15 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Repool db1034 - T160390 (duration: 00m 38s) |
[production] |
| 06:07 |
<marostegui> |
Deploy schema change db1034 (s7) - T160390 |
[production] |
| 06:03 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Add tempdb2001 to x1 as a slave - T162290 (duration: 00m 38s) |
[production] |
| 06:01 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Depool db1034 - T160390 (duration: 00m 39s) |
[production] |
| 02:49 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Mon Apr 10 02:49:06 UTC 2017 (duration 5m 40s) |
[production] |
| 02:43 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.19) (duration: 07m 32s) |
[production] |
| 02:23 |
<l10nupdate@tin> |
scap sync-l10n completed (1.29.0-wmf.18) (duration: 08m 17s) |
[production] |
|
2017-04-08
§
|
| 20:56 |
<bblack> |
removed varnishkafka logs and daemon.log.1 on cp1052 to free disk space and clear alert |
[production] |
| 17:43 |
<chasemp> |
service nova-compute restart labvirt1002 |
[production] |
| 17:36 |
<chasemp> |
nova reset-state on 15 nodepool stuck in deletion nodes, and force-delete |
[production] |
| 17:29 |
<chasemp> |
delete manual on labcontrol all instances in delete state on nodepool |
[production] |
| 17:25 |
<chasemp> |
openstack server delete 970a86ce-2549-4cf3-be91-1f8558ab1b32 (admin-monitoring stuck in build) |
[production] |
| 17:21 |
<chasemp> |
restart rabbitmq on labcontrol1001 |
[production] |
| 17:20 |
<chasemp> |
restart nova-api on labnet |
[production] |
| 16:00 |
<bblack> |
banning obj.http.Content-Type ~ text/html on cache_upload |
[production] |
| 15:46 |
<bblack> |
banning obj.http.X-Orig-Content-Type !~ . on cache_upload in ulsfo |
[production] |
| 14:56 |
<bblack> |
banning obj.http.X-Orig-Content-Type !~ . on cache_upload in esams |
[production] |
| 13:54 |
<bblack> |
banning obj.http.X-Orig-Content-Type !~ . on cache_upload in codfw |
[production] |
| 13:27 |
<bblack> |
banning obj.http.X-Orig-Content-Type !~ . on cache_upload in eqiad |
[production] |
| 11:55 |
<bblack> |
banning obj.http.Content-Type ~ text/html on cache_upload |
[production] |
| 10:55 |
<jynus> |
setting labsdb1001 and labsdb1003 in read only mode |
[production] |