|
2019-07-03
ยง
|
| 13:54 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:54 |
<XioNoX> |
remove all mentions of sampling (curently disabled) on cr2-esams to try to reduce memory usage |
[production] |
| 13:51 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:51 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:33 |
<moritzm> |
rebooting doc1001 to pick up MDS-enabled qemu |
[production] |
| 13:30 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 13:30 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 13:24 |
<jynus> |
upgrade and restart db2097 T225378 |
[production] |
| 13:08 |
<ema> |
depool cp1076 and reimage as upload_ats T226638 |
[production] |
| 13:07 |
<ema> |
depool cp1076 and reimage as upload_ats T226637 |
[production] |
| 12:55 |
<marostegui> |
Drop secret and stratch_tokens columns from centralauth (s7) T226826 |
[production] |
| 12:53 |
<ema> |
pool cp2026 w/ ATS backend T226637 |
[production] |
| 12:50 |
<Urbanecm> |
foreachwiki refreshImageMetadata.php --mediatype=AUDIO --mime=audio/mid --force completed (T226784) |
[production] |
| 12:40 |
<Urbanecm> |
Started foreachwiki refreshImageMetadata.php --mediatype=AUDIO --mime=audio/mid --force for T226784 on mwmaint1002 in a tmux |
[production] |
| 12:40 |
<moritzm> |
rebooting mendelevium (ticket.wikimedia.org) to pick up MDS-enabled qemu |
[production] |
| 12:39 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:39 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:35 |
<moritzm> |
rebooting dubnium/pollux (corp LDAP replicas) to pick up MDS-enabled qemu |
[production] |
| 12:34 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:34 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:31 |
<moritzm> |
rebooting neon (kubernetes staging master) to pick up MDS-enabled qemu |
[production] |
| 12:30 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:30 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:24 |
<moritzm> |
rebooting bromine to pick up MDS-enabled qemu |
[production] |
| 12:24 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:24 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:21 |
<moritzm> |
rebooting pybal-test hosts to pick up MDS-enabled qemu |
[production] |
| 12:19 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:19 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:14 |
<ema> |
reimage cp2026 as upload_ats T226637 |
[production] |
| 12:12 |
<kart_> |
Updated cxserver to b447674 (T226611) |
[production] |
| 12:10 |
<kartik@deploy1001> |
scap-helm cxserver finished |
[production] |
| 12:10 |
<kartik@deploy1001> |
scap-helm cxserver cluster eqiad completed |
[production] |
| 12:10 |
<kartik@deploy1001> |
scap-helm cxserver upgrade -f cxserver-eqiad-values.yaml production stable/cxserver [namespace: cxserver, clusters: eqiad] |
[production] |
| 12:09 |
<kartik@deploy1001> |
scap-helm cxserver finished |
[production] |
| 12:09 |
<kartik@deploy1001> |
scap-helm cxserver cluster codfw completed |
[production] |
| 12:09 |
<kartik@deploy1001> |
scap-helm cxserver upgrade -f cxserver-codfw-values.yaml production stable/cxserver [namespace: cxserver, clusters: codfw] |
[production] |
| 12:07 |
<kartik@deploy1001> |
scap-helm cxserver finished |
[production] |
| 12:07 |
<kartik@deploy1001> |
scap-helm cxserver cluster staging completed |
[production] |
| 12:07 |
<kartik@deploy1001> |
scap-helm cxserver upgrade -f cxserver-staging-values.yaml staging stable/cxserver [namespace: cxserver, clusters: staging] |
[production] |
| 11:55 |
<reedy@deploy1001> |
Synchronized php-1.34.0-wmf.11/extensions/TimedMediaHandler/: T226840 (duration: 00m 50s) |
[production] |
| 11:29 |
<moritzm> |
ran puppet clean/deactivate and debdeploy removal for cp3037 (host is broken for a long time and triggering failing Cumin/debdeploy runs) T227077 |
[production] |
| 11:14 |
<Urbanecm> |
EU SWAT done |
[production] |
| 11:14 |
<Urbanecm> |
Ran mwscript namespaceDupes.php --wiki=pawikisource --fix for T226959 |
[production] |
| 11:12 |
<urbanecm@deploy1001> |
Synchronized wmf-config/throttle.php: SWAT: [[:gerrit:520408|Add new throttle rule for enwiki event]] (T227059) (duration: 00m 48s) |
[production] |
| 11:11 |
<urbanecm@deploy1001> |
Synchronized wmf-config/throttle-analyze.php: SWAT: [[:gerrit:518298|[throttle-analyze] Grant autoconfirmed permission to user when throttle rule is applied]] (T204583) (duration: 00m 49s) |
[production] |
| 11:11 |
<moritzm> |
rebooting people1001 (people.wikimedia.org) to pick up MDS-enabled qemu |
[production] |
| 11:06 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:520174|Configuring Namespaces at pawikisource]] (T226959) (duration: 00m 52s) |
[production] |
| 11:05 |
<moritzm> |
rebooting krypton nodes to pick up MDS-enabled qemu |
[production] |
| 11:05 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |