|
2019-07-15
§
|
| 10:34 |
<jdrewniak@deploy1001> |
Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:523127| Bumping portals to master (T128546)]] (duration: 00m 56s) |
[production] |
| 09:56 |
<ema> |
cp-eqsin: varnish frontend rolling restarts for 5.1.3-1wm11 upgrades T227672 |
[production] |
| 09:39 |
<fsero> |
repooling ms-fe2005 T227570 |
[production] |
| 08:50 |
<fsero> |
creating docker_registry_codfw on eqiad T227570 |
[production] |
| 08:49 |
<gehel> |
correction: set oemhp_powerreg=os + reboot for elastic1052 (NOT elastic1054) - T225713 |
[production] |
| 08:48 |
<fsero> |
T227570 changing container_synchronization on docker_registry_codfw to //docker_registry/eqiad/AUTH_docker/docker_registry_codfw |
[production] |
| 08:48 |
<gehel> |
set oemhp_powerreg=os + reboot for elastic1054 - T225713 |
[production] |
| 08:22 |
<godog> |
set oemhp_powerreg=os on ms-be10[16-39] - T225713 |
[production] |
| 08:01 |
<vgutierrez> |
upgrading acme-chief to version 0.19 in acme-chief production instances - T225945 |
[production] |
|
2019-07-12
§
|
| 23:35 |
<mutante> |
netmon1003 - shutdown -h now after it's gone from Icinga now |
[production] |
| 23:31 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 23:31 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 23:28 |
<mutante> |
netmon1003 - stopping apache2 service (decom of servermon.wikimedia.org) |
[production] |
| 19:41 |
<James_F> |
Disabled 2FA for MSchottlender-WMF for device reset. |
[production] |
| 19:17 |
<shdubsh> |
add prometheus-varnishkafka-exporter 0.1 to apt repo T196066 |
[production] |
| 19:15 |
<urandom> |
bootstrapping restbase1017-c -- T222960 |
[production] |
| 19:08 |
<jeh> |
rebooting cloudvirt1018.eqiad.wmnet T216040 |
[production] |
| 18:53 |
<mutante> |
cp1072 - enabling notifications for service checks in icinga, they were disabled but all green and no SAL/ticket. looked like forgotten from the past |
[production] |
| 18:49 |
<gehel> |
setting CPU governor to performance for wdqs1010 - T225713 |
[production] |
| 18:16 |
<Krinkle> |
Remove bogus Graphite data at frontend.navtiming2.requet (typo from Nov 2018), graphite1004/2003 |
[production] |
| 18:02 |
<urandom> |
bootstrapping restbase1017-b -- T222960 |
[production] |
| 16:32 |
<urandom> |
bootstrapping restbase1017-a -- T222960 |
[production] |
| 16:25 |
<jijiki> |
Rolling restart swift proxy on ms-fe* |
[production] |
| 15:25 |
<jeh> |
rebooting cloudvirt1018.eqiad.wmnet T216040 |
[production] |
| 14:05 |
<gehel@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 12:45 |
<gehel@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 12:39 |
<fsero> |
recreating ci staging namespaces T227775 |
[production] |
| 12:39 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'blubberoid' for release 'staging' . |
[production] |
| 12:38 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'blubberoid' for release 'staging' . |
[production] |
| 12:36 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-main' for release 'main' . |
[production] |
| 12:33 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . |
[production] |
| 12:33 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-analytics' for release 'analytics' . |
[production] |
| 12:22 |
<fsero> |
recreating eventgate-* and blubberoid staging namespaces T227775 |
[production] |
| 12:22 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'mathoid' for release 'staging' . |
[production] |
| 12:22 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'mathoid' for release 'staging' . |
[production] |
| 12:18 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 12:18 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 12:18 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 12:15 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'sessionstore' for release 'staging' . |
[production] |
| 12:11 |
<fsero> |
recreating sessionstore,cxserver and mathoid staging namespaces T227775 |
[production] |
| 12:10 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'citoid' for release 'staging' . |
[production] |
| 12:06 |
<fsero> |
recreating citoid staging namespace T227775 |
[production] |
| 12:05 |
<fsero@> |
helmfile [STAGING] Ran 'apply' command on namespace 'termbox' for release 'staging' . |
[production] |
| 12:01 |
<fsero> |
recreating termbox staging namespace T227775 |
[production] |
| 11:09 |
<jynus@deploy1001> |
Synchronized wmf-config/db-codfw.php: Switchover db2045 x1 codfw master to db2069 (duration: 00m 51s) |
[production] |
| 10:24 |
<jynus> |
switchover x1 codfw master from db2045 to db2069 T227862 |
[production] |
| 10:23 |
<jynus> |
switchover x1 codfw master from db2045 to db2069 |
[production] |