|
2019-02-21
§
|
| 12:00 |
<arturo> |
disable puppet in install1002 to test T216497 |
[production] |
| 11:13 |
<volans> |
upgraded spicerack to 0.0.19 on cumin[12]001 |
[production] |
| 11:11 |
<volans> |
uploaded spicerack_0.0.19-1_amd64.deb to apt.wikimedia.org stretch-wikimedia |
[production] |
| 10:55 |
<akosiaris> |
upgrade mathoid staging+production to latest helm chart |
[production] |
| 10:47 |
<akosiaris@deploy1001> |
scap-helm mathoid finished |
[production] |
| 10:47 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster codfw completed |
[production] |
| 10:47 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster eqiad completed |
[production] |
| 10:47 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade -f mathoid-values.yaml production stable/mathoid [namespace: mathoid, clusters: eqiad,codfw] |
[production] |
| 10:29 |
<akosiaris@deploy1001> |
scap-helm mathoid finished |
[production] |
| 10:29 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster staging completed |
[production] |
| 10:29 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade -f mathoid-staging-values.yaml staging stable/mathoid [namespace: mathoid, clusters: staging] |
[production] |
| 10:28 |
<akosiaris@deploy1001> |
scap-helm mathoid finished |
[production] |
| 10:28 |
<akosiaris@deploy1001> |
scap-helm mathoid cluster staging completed |
[production] |
| 10:28 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade -f mathoid-values.yaml staging stable/mathoid [namespace: mathoid, clusters: staging] |
[production] |
| 10:27 |
<akosiaris@deploy1001> |
scap-helm mathoid upgrade -f mathoid-values.yaml stable/mathoid [namespace: mathoid, clusters: staging] |
[production] |
| 10:26 |
<akosiaris@deploy1001> |
scap-helm list finished |
[production] |
| 10:26 |
<akosiaris@deploy1001> |
scap-helm list cluster codfw completed |
[production] |
| 10:26 |
<akosiaris@deploy1001> |
scap-helm list cluster eqiad completed |
[production] |
| 10:26 |
<akosiaris@deploy1001> |
scap-helm list [namespace: list, clusters: eqiad,codfw] |
[production] |
| 10:23 |
<godog> |
on boron unblock trusty builds with umount /var/cache/pbuilder/base-trusty-amd64.cow/dev/ptmx |
[production] |
| 10:04 |
<akosiaris> |
create citoid namespace on kubernetes eqiad codfw staging clusters T213194 |
[production] |
| 10:04 |
<akosiaris> |
create cxserver namespace on kubernetes eqiad codfw staging clusters T213195 |
[production] |
| 09:35 |
<volans> |
force rebooting unresponsive icinga1001 T214760 |
[production] |
| 09:29 |
<marostegui> |
Deploy schema change on s3 primary master (db1078) - T210713 |
[production] |
| 09:29 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1075 T210713 (duration: 00m 52s) |
[production] |
| 09:14 |
<moritzm> |
temporarily stop prometheus@labs.service on labmon for journald restarts (part of security update) |
[production] |
| 08:40 |
<marostegui> |
Deploy schema change on db1075 - T210713 |
[production] |
| 08:40 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1075 T210713 (duration: 00m 54s) |
[production] |
| 08:35 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1077 T210713 (duration: 00m 53s) |
[production] |
| 07:44 |
<moritzm> |
rolling out remaining systemd security updates on jessie |
[production] |
| 07:12 |
<marostegui> |
Deploy schema change on db1077 - this will generate lag on labsdb:s3 T210713 |
[production] |
| 07:12 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1077 T210713 (duration: 00m 56s) |
[production] |
| 07:08 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Repool db1123 T210713 (duration: 00m 55s) |
[production] |
| 06:22 |
<marostegui> |
Deploy schema change on db1123 - T210713 |
[production] |
| 06:21 |
<marostegui@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Depool db1123 T210713 (duration: 00m 57s) |
[production] |
| 05:46 |
<bblack> |
repooling cp5010 - T214274 |
[production] |
| 05:41 |
<bblack> |
removing cp5010 downtimes from icinga - T214274 |
[production] |
| 05:34 |
<bblack> |
rebooting cp5010 for device name on swapped disk (depooled) - T214274 |
[production] |
| 04:30 |
<kart_> |
Finished: Fifth manual run of unpublished draft purge script for ContentTranslation (T216470) |
[production] |
| 04:16 |
<XioNoX> |
Unplug Tata/NTT/PCCW from cr1-eqsin - T213121 |
[production] |
| 03:21 |
<XioNoX> |
replace cp5010 disk 1 - T214274 |
[production] |
| 03:15 |
<kart_> |
Fifth manual run of unpublished draft purge script for ContentTranslation (T216470) |
[production] |
| 02:44 |
<XioNoX> |
depool eqsin - T213121 |
[production] |
| 02:31 |
<twentyafterfour> |
phabricator upgrade finished, service appears to be returned to normal |
[production] |
| 01:43 |
<twentyafterfour> |
running phabricator database schema changes |
[production] |
| 01:38 |
<twentyafterfour> |
now taking phabricator offline for upgrade |
[production] |
| 01:15 |
<twentyafterfour> |
Taking phabricator offline momentarily for upgrade |
[production] |
| 01:01 |
<twentyafterfour> |
set downtime in icinga for phab100* |
[production] |
| 00:17 |
<catrope@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Enable partial blocks on metawiki and mediawikiwiki (T216065) (duration: 00m 54s) |
[production] |