|
2021-12-17
ยง
|
| 15:34 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality' for release 'main' . |
[production] |
| 15:33 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . |
[production] |
| 15:16 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@5c3bce1]: Fix refine sanitize allowlist, remove mediawiki_skin_diff schema for now |
[production] |
| 14:52 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-main2001.codfw.wmnet with OS buster |
[production] |
| 14:35 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@e9f04c3] (hadoop-test): Fix sanitize allowlist problem [TEST] (duration: 69m 41s) |
[production] |
| 14:22 |
<elukey@cumin1001> |
START - Cookbook sre.hosts.reimage for host kafka-main2001.codfw.wmnet with OS buster |
[production] |
| 13:25 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@e9f04c3] (hadoop-test): Fix sanitize allowlist problem [TEST] |
[production] |
| 13:20 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@e9f04c3] (thin): Fix sanitize allowlist problem [THIN] (duration: 00m 07s) |
[production] |
| 13:20 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@e9f04c3] (thin): Fix sanitize allowlist problem [THIN] |
[production] |
| 13:20 |
<milimetric@deploy1002> |
Finished deploy [analytics/refinery@e9f04c3]: Fix sanitize allowlist problem (duration: 26m 19s) |
[production] |
| 13:06 |
<mmandere@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host bast6001.wikimedia.org |
[production] |
| 13:04 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=druid1008.eqiad.wmnet,service=druid-public-broker |
[production] |
| 13:02 |
<btullis> |
upgraded druid packages on druid1008 |
[production] |
| 13:01 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=druid1008.eqiad.wmnet,service=druid-public-broker |
[production] |
| 13:00 |
<btullis> |
upgraded druid packages on an-druid1005 |
[production] |
| 12:58 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=druid1007.eqiad.wmnet,service=druid-public-broker |
[production] |
| 12:55 |
<btullis> |
upgrading druid packages on druid1007 |
[production] |
| 12:54 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=druid1007.eqiad.wmnet,service=druid-public-broker |
[production] |
| 12:53 |
<milimetric@deploy1002> |
Started deploy [analytics/refinery@e9f04c3]: Fix sanitize allowlist problem |
[production] |
| 12:52 |
<btullis> |
upgrading druid packages on an-druid1004 |
[production] |
| 12:51 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=druid1006.eqiad.wmnet,service=druid-public-broker |
[production] |
| 12:50 |
<btullis> |
upgrading druid packages on druid1006 |
[production] |
| 12:48 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=druid1006.eqiad.wmnet,service=druid-public-broker |
[production] |
| 12:47 |
<btullis> |
Upgrading druid packages on an-druid1003 |
[production] |
| 12:44 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=druid1005.eqiad.wmnet,service=druid-public-broker |
[production] |
| 12:43 |
<btullis> |
Upgraded druid packages on druid1005. |
[production] |
| 12:43 |
<btullis> |
Upgraded druid packages on an-druid1002. |
[production] |
| 12:43 |
<mmandere@cumin1001> |
START - Cookbook sre.ganeti.makevm for new host bast6001.wikimedia.org |
[production] |
| 12:41 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=druid1005.eqiad.wmnet,service=druid-public-broker |
[production] |
| 12:40 |
<mmandere@cumin1001> |
END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host bast6001.wikimedia.org |
[production] |
| 12:37 |
<btullis> |
upgrading druid packages on an-druid1002 |
[production] |
| 12:32 |
<mmandere@cumin1001> |
START - Cookbook sre.ganeti.makevm for new host bast6001.wikimedia.org |
[production] |
| 12:31 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=druid1004.eqiad.wmnet,service=druid-public-broker |
[production] |
| 12:31 |
<btullis> |
Upgraded druid packages on druid1004 |
[production] |
| 12:07 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=no; selector: name=druid1004.eqiad.wmnet,service=druid-public-broker |
[production] |
| 11:42 |
<btullis> |
Upgrading druid packages on an-druid1001. |
[production] |
| 11:18 |
<btullis> |
updating reprepro with new druid packages for buster-wikimedia to pick up new log4j jar files |
[production] |
| 10:26 |
<btullis@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=aqs1010.eqiad.wmnet |
[production] |
| 09:57 |
<jelto@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 09:57 |
<jelto@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 09:54 |
<elukey@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-main2002.codfw.wmnet with OS buster |
[production] |
| 09:51 |
<jelto@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 09:50 |
<jelto@deploy1002> |
helmfile [staging-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 09:37 |
<dcausse> |
restart blazegraph on wdqs1007 (jvm stuck for 6hours) |
[production] |
| 09:23 |
<elukey@cumin1001> |
START - Cookbook sre.hosts.reimage for host kafka-main2002.codfw.wmnet with OS buster |
[production] |
| 09:04 |
<godog> |
previous message refers to ms-be2065 |
[production] |
| 09:03 |
<godog> |
set sdq as offline, showing errors. megacli -PDOffline -PhysDrv '[32:14]' -aALL |
[production] |
| 08:50 |
<oblivian@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 07:30 |
<oblivian@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 07:03 |
<oblivian@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |