|
2020-02-14
§
|
| 09:05 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:05 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:05 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 09:05 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 09:05 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 08:46 |
<moritzm> |
installing 4.19.98 kernel update on Buster systems |
[production] |
| 08:06 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repool db1107 with weight 100 for 10.4 testing - T242702', diff saved to https://phabricator.wikimedia.org/P10408 and previous config saved to /var/cache/conftool/dbconfig/20200214-080600-marostegui.json |
[production] |
| 06:51 |
<vgutierrez> |
updating puppet compiler facts |
[production] |
| 01:27 |
<dpifke@deploy1001> |
Finished deploy [performance/navtiming@2eec00a]: (no justification provided) (duration: 00m 05s) |
[production] |
| 01:27 |
<dpifke@deploy1001> |
Started deploy [performance/navtiming@2eec00a]: (no justification provided) |
[production] |
| 00:53 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T245202 cirrus: Move all move_like traffic to codfw (duration: 01m 02s) |
[production] |
| 00:51 |
<jforrester@deploy1001> |
Synchronized wmf-config/PoolCounterSettings.php: T245202 cirrus: Increase the pool counter limits a bit (duration: 01m 05s) |
[production] |
| 00:38 |
<bd808> |
Added tools-k8s-worker-35 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:34 |
<bd808> |
Added tools-k8s-worker-34 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:32 |
<bd808> |
Added tools-k8s-worker-33 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:29 |
<bd808> |
Added tools-k8s-worker-32 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:25 |
<bd808> |
Added tools-k8s-worker-31 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:25 |
<bd808> |
Added tools-k8s-worker-30 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:17 |
<bd808> |
Added tools-k8s-worker-29 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:15 |
<bd808> |
Added tools-k8s-worker-28 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:13 |
<bd808> |
Added tools-k8s-worker-27 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:07 |
<bd808> |
Added tools-k8s-worker-26 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 00:03 |
<bd808> |
Added tools-k8s-worker-25 to 2020 Kubernetes cluster (T244791) |
[tools] |
|
2020-02-13
§
|
| 23:53 |
<bd808> |
Added tools-k8s-worker-24 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 23:50 |
<bd808> |
Added tools-k8s-worker-23 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 23:38 |
<bd808> |
Added tools-k8s-worker-22 to 2020 Kubernetes cluster (T244791) |
[tools] |
| 22:19 |
<James_F> |
Zuul: Add *.microsoft.com in email_whitelist |
[releng] |
| 22:13 |
<jeh> |
running filesystem tests on cloudvirt1024 T241884 |
[production] |
| 21:57 |
<Framawiki> |
edited crons to use new specific py2 pwb.py endpoint since T213287 |
[tools.totoazero] |
| 21:42 |
<otto@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'canary' . |
[production] |
| 21:41 |
<otto@deploy1001> |
helmfile [EQIAD] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'production' . |
[production] |
| 21:40 |
<jbond42> |
refresh facts on compilers |
[production] |
| 21:38 |
<otto@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'canary' . |
[production] |
| 21:37 |
<otto@deploy1001> |
helmfile [CODFW] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'production' . |
[production] |
| 21:35 |
<ottomata> |
deploying production and canary releases for eventgate-logging-external (and destroying the 'logging-external' release) (safe because eventgate-logging-external is not in use) - T245203 |
[production] |
| 21:35 |
<bd808> |
Deleted tools-sgewebgrid-lighttpd-092{1,2,3,4,5,6,7,8} & tools-sgewebgrid-generic-090{3,4} (T244791) |
[tools] |
| 21:33 |
<bd808> |
Removed tools-sgewebgrid-lighttpd-092{1,2,3,4,5,6,7,8} & tools-sgewebgrid-generic-090{3,4} from grid engine config (T244791) |
[tools] |
| 21:29 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'canary' . |
[production] |
| 21:28 |
<otto@deploy1001> |
helmfile [STAGING] Ran 'apply' command on namespace 'eventgate-logging-external' for release 'production' . |
[production] |
| 20:33 |
<marxarelli> |
rollback to group1 due to 500 spike (2k/min) (T233867) |
[production] |
| 20:32 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: (no justification provided) |
[production] |
| 20:30 |
<marxarelli> |
varnish 500 spike. rolling back |
[production] |
| 20:20 |
<gehel> |
restarting blazegraph + updater on wdqs2006 |
[production] |
| 20:19 |
<dduvall@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.35.0-wmf.19 |
[production] |
| 19:44 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.19/includes/api/ApiRollback.php: T245159 ApiRollback: Properly deal with UserIdentity (duration: 01m 04s) |
[production] |
| 19:20 |
<jforrester@deploy1001> |
Synchronized php-1.35.0-wmf.19/includes/resourceloader/ResourceLoaderSkinModule.php: T245182 ResourceLoaderSkinModule: Don't hard-deprecate wgLogoHD just now (duration: 01m 03s) |
[production] |
| 19:17 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T219534 Add new MLR models for Cirrus on zh/ja/kowiki (duration: 01m 03s) |
[production] |
| 19:13 |
<bd808> |
Restarting to pick up latest ingress configuration |
[tools.replag] |
| 19:10 |
<moritzm> |
installing e2fsprogs security updates |
[production] |
| 19:02 |
<Reedy> |
is down "Hound is still starting up, please wait a few minutes for the initial indexing to complete." |
[codesearch] |