|
2020-11-10
ยง
|
| 23:19 |
<dpifke> |
Cherry-picking https://gerrit.wikimedia.org/r/c/operations/puppet/+/640226 and https://gerrit.wikimedia.org/r/c/performance/coal/+/640227 in beta; should only affect deployment-webperf11. |
[releng] |
| 22:14 |
<mholloway-shell@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'nontls' . |
[production] |
| 22:14 |
<mholloway-shell@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 22:13 |
<thcipriani> |
restarting php7.2-fpm on deployment-jobrunner03 and deployment-parsoid11 |
[releng] |
| 22:08 |
<mholloway-shell@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'nontls' . |
[production] |
| 22:08 |
<mholloway-shell@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 22:05 |
<mholloway-shell@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |
| 21:59 |
<mholloway-shell@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 21:58 |
<jgleeson> |
update civicrm revision changed from c36a5cc1b1 to d0cd7f6dbb |
[production] |
| 21:57 |
<mholloway-shell@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 21:55 |
<mholloway-shell@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'wikifeeds' for release 'staging' . |
[production] |
| 21:47 |
<ebernhardson> |
unban elastic1050 from eqiad search psi cluster |
[production] |
| 21:28 |
<cstone> |
civicrm revision changed from b1342c4129 to c36a5cc1b1 |
[production] |
| 21:24 |
<brennen@deploy1001> |
sync-file aborted: Testing: README.md sync-file with ssh -n for T223287 (duration: 00m 37s) |
[production] |
| 21:23 |
<brennen> |
testing some scap operations, modified to use ssh -n for debugging T223287 |
[production] |
| 21:11 |
<ebernhardson> |
ban elastic1050 from eqiad psi cluster due to excessive load |
[production] |
| 21:02 |
<brennen@deploy1001> |
Finished scap: Backport: [[gerrit:640487|language: Honor $wgTranslateNumerals, even if PHP does digit translation(T267614)]] and [[gerrit:640488|Downgrade the severity of the non-numeric argument to formatNum warnings (T267370, T267587)]] (duration: 34m 46s) |
[production] |
| 20:27 |
<brennen@deploy1001> |
Started scap: Backport: [[gerrit:640487|language: Honor $wgTranslateNumerals, even if PHP does digit translation(T267614)]] and [[gerrit:640488|Downgrade the severity of the non-numeric argument to formatNum warnings (T267370, T267587)]] |
[production] |
| 20:16 |
<chicocvenancio> |
restart hub to apply move to sqlite. T267667 |
[paws] |
| 20:10 |
<brennen@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:640254|Turn on formatnum logging (T267587, T267370)]] (duration: 01m 02s) |
[production] |
| 19:45 |
<andrewbogott> |
rebooting tools-sgeexec-0950; OOM |
[tools] |
| 19:42 |
<bstorm> |
safelisted "argocd" namespace with namespaceSelector for registry-admission controller |
[toolsbeta] |
| 19:32 |
<joal> |
Deploy wikistats2 v2.8.2 |
[analytics] |
| 19:06 |
<hknust> |
holger mwmaint1002 Stop T219279 |
[production] |
| 18:49 |
<legoktm> |
associated floating IP to toolsbeta-docker-registry-01 and pointed DNS docker-registry.toolsbeta.wmflabs.org. at it |
[toolsbeta] |
| 18:40 |
<apergos> |
deployment-prep: switched deployment-prep to libicu63, see T264991 |
[releng] |
| 18:31 |
<hknust> |
holger mwmaint1002 Start T219279 |
[production] |
| 18:27 |
<legoktm> |
creating toolsbeta-docker-imagebuilder-01 (T267616) |
[toolsbeta] |
| 18:16 |
<joal> |
Releasing refinery-source v0,0,139 to archiva |
[analytics] |
| 17:57 |
<effie> |
pool mw1263 mw1264 |
[production] |
| 17:31 |
<effie> |
briefly depool mw1263 and mw1264 |
[production] |
| 17:30 |
<jynus> |
about to shutdown db1139 for hw maintenance T261405 |
[production] |
| 17:18 |
<dcaro> |
launching instance toolsbeta-test-k8s-etcd-4 (T267140) |
[toolsbeta] |
| 17:15 |
<dcaro> |
removing unused toolsbeta-k8s-etcd prefix (we use toolsbeta-test-k8s-etcd) (T267140) |
[toolsbeta] |
| 17:13 |
<dwisehaupt> |
upping thank you mail flow through frmx's to 30% of the total runs |
[production] |
| 16:41 |
<arturo> |
set paws in sqlite mode because T266587 (kubectl --namespace prod edit configmap hub-config) |
[paws] |
| 16:37 |
<arturo> |
icinga downtime toolschecker for 2h becasue toolsdb maintenance (T266587) |
[admin] |
| 16:32 |
<XioNoX> |
add cloud-storage1-b-codfw to, well, codfw switches - T267378 |
[production] |
| 16:24 |
<bstorm> |
set toolsdb to read-only T266587 |
[clouddb-services] |
| 16:20 |
<effie> |
pool mw1263 |
[production] |
| 16:17 |
<hashar> |
Restarting Gerrit on gerrit1001 |
[production] |
| 16:12 |
<hashar> |
Restarted Gerrit on gerrit2001 for config change |
[production] |
| 15:57 |
<wm-bot> |
<lucaswerkmeister> made tool readonly to avoid issues during ToolsDB maintenance |
[tools.quickcategories] |
| 15:53 |
<zpapierski@deploy1001> |
Finished deploy [wikimedia/discovery/analytics@1ab89ed]: Deploying venv workaround for Debian 9 (duration: 01m 06s) |
[production] |
| 15:52 |
<zpapierski@deploy1001> |
Started deploy [wikimedia/discovery/analytics@1ab89ed]: Deploying venv workaround for Debian 9 |
[production] |
| 15:38 |
<moritzm> |
installing 4.19.152 kernel packages on buster hosts (only installing the package, reboots will happen separately) |
[production] |
| 15:28 |
<effie> |
depool mw1263 - T244340 |
[production] |
| 15:09 |
<ejegg> |
updated fundraising python tools from 087a596d3a to 7853f426ee |
[production] |
| 14:48 |
<mforns> |
restarted data quality stats daily bundle with new metric |
[analytics] |
| 14:44 |
<dcaro> |
taking down one of the test-k8s etcd nodes to reimage (T267140) |
[toolsbeta] |