|
2018-10-27
§
|
| 21:24 |
<banyek> |
resetting power on db1117 as the host is DOWN and the serial console shows nothing |
[production] |
| 20:56 |
<onimisionipe> |
depooling wdqs1003 to catch up with others |
[production] |
| 17:47 |
<Nettrom> |
apt-get installed python3-dev libmariadbclient-dev on suggestbot-01 |
[suggestbot] |
| 17:37 |
<Nettrom> |
apt-get installed python-pip, python-dev, and build-essentials, then installed virtualenv globally via pip on suggestbot-01 |
[suggestbot] |
| 17:29 |
<Nettrom> |
created suggestbot database and user on suggestbot-01, restricted connections to localhost |
[suggestbot] |
| 16:38 |
<Nettrom> |
installed MariaDB on suggestbot-01 |
[suggestbot] |
| 16:31 |
<Nettrom> |
launched suggestbot-01 instance on eqiad1-r |
[suggestbot] |
| 16:18 |
<addshore@deploy1001> |
Synchronized wmf-config/Wikibase.php: Wikibase, fix duplicate specialSiteLinkGroups key T208124 (duration: 00m 54s) |
[production] |
| 15:57 |
<addshore@deploy1001> |
Synchronized wmf-config/Wikibase.php: Wikibase, make sure specialSiteLinkGroups has wikidata group (duration: 00m 54s) |
[production] |
| 12:32 |
<Amir1> |
Deployed patch for T207576 |
[production] |
| 12:29 |
<banyek> |
resuming replication on s1@dbstore2002 as table compression is finished (T204930) |
[production] |
| 09:17 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: BETA ONLY (4x patches) (duration: 00m 55s) |
[production] |
| 09:09 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: Remove wgArticlePlaceholderSearchIntegrationBackend BETA override (duration: 01m 00s) |
[production] |
| 08:34 |
<addshore@deploy1001> |
Synchronized wmf-config/Wikibase.php: Wikibase, Set siteLinkGroups settings on all wikis again T208048 T208077 T208074 (duration: 00m 54s) |
[production] |
| 08:24 |
<addshore@deploy1001> |
Synchronized wmf-config/InitialiseSettings-labs.php: BETA ONLY T208043 (duration: 01m 06s) |
[production] |
| 03:41 |
<SMalyshev> |
depool wdqs1003 again to let it catch up some more |
[production] |
| 02:35 |
<smalyshev@deploy1001> |
Finished deploy [wdqs/wdqs@7eeede7]: Re-deploy Updater to deal with performance issues (duration: 00m 38s) |
[production] |
| 02:34 |
<smalyshev@deploy1001> |
Started deploy [wdqs/wdqs@7eeede7]: Re-deploy Updater to deal with performance issues |
[production] |
| 02:34 |
<smalyshev@deploy1001> |
Finished deploy [wdqs/wdqs@e9392f4]: Re-deploy Updater to deal with performance issues (duration: 00m 05s) |
[production] |
| 02:33 |
<smalyshev@deploy1001> |
Started deploy [wdqs/wdqs@e9392f4]: Re-deploy Updater to deal with performance issues |
[production] |
| 00:00 |
<mutante> |
icinga1001 - using wmf-auto-reimage to reinstall gets stuck at initial puppet run after reboot - Still waiting for Puppet after 105.0 minutes - aborting on cumin, loggin in directly and manually running puppet (T202782 T208100) |
[production] |
|
2018-10-26
§
|
| 22:54 |
<mutante> |
sodium - attempted to replace broken disk for RAID - did not go well |
[production] |
| 21:38 |
<ejegg> |
updated fundraising CiviCRM from 97506677e8 to 65130ef3dd |
[production] |
| 21:36 |
<aaron@deploy1001> |
Synchronized php-1.33.0-wmf.1/includes: 86c0b56b0d1bf66073fafb9bc00bafb87d2e3b9c (duration: 01m 14s) |
[production] |
| 21:34 |
<aaron@deploy1001> |
Synchronized php-1.33.0-wmf.1/autoload.php: 86c0b56b0d1bf66073fafb9bc00bafb87d2e3b9c (duration: 00m 52s) |
[production] |
| 21:33 |
<aaron@deploy1001> |
Synchronized php-1.33.0-wmf.1/tests: 86c0b56b0d1bf66073fafb9bc00bafb87d2e3b9c (duration: 01m 08s) |
[production] |
| 20:03 |
<mutante> |
icinga1001 - disabled puppet, changed: check_result_reaper_frequency=2 ; max_check_result_reaper_time=10 to test if it lowers latency (T208066) |
[production] |
| 19:40 |
<chasemp> |
remove 2fa for charlottepotero and cwd users in phab (so they can readd) |
[production] |
| 19:09 |
<SMalyshev> |
repooled wdqs1003 - looks like it caught up now |
[production] |
| 18:33 |
<andrewbogott> |
region migration finished |
[analytics] |
| 17:18 |
<SMalyshev> |
depool wdqs1003 again to let it catch up some more |
[production] |
| 16:10 |
<ejegg> |
updated payments-wiki to 34506ce636 |
[production] |
| 15:32 |
<elukey> |
rolling restart of all prometheus-mcrouter-exporters on app/api servers - metrics not reported after the last mcrouter restart |
[production] |
| 15:20 |
<gehel> |
repooling wdqs1003, other nodes are starting to lag as well |
[production] |
| 14:56 |
<reedy@deploy1001> |
Synchronized wmf-config/CommonSettings.php: Prevent sysops from disabling 2FA for other users as part of upcoming feature (duration: 00m 53s) |
[production] |
| 13:36 |
<andrewbogott> |
migrating project to eqiad1 |
[analytics] |
| 13:02 |
<gehel> |
depool wdqs1003 to catch up on updates |
[production] |
| 11:49 |
<joal> |
Rerun failed oozie jobs (pageview and projectview) |
[analytics] |
| 10:34 |
<arturo> |
T207970 added misctools 1.31 and jobutils 1.30 to stretch-tools aptly repo |
[tools] |
| 10:32 |
<arturo> |
T209970 added misctools 1.31 and jobutils 1.30 to stretch-tools aptly repo |
[tools] |
| 09:04 |
<hashar> |
jenkins: update IRC config at https://integration.wikimedia.org/ci/configure to replace #wikimedia-ri-ci by #wikimedia-infrastructure ( https://gerrit.wikimedia.org/r/#/c/integration/config/+/469034/ ) |
[releng] |
| 07:51 |
<bawolff> |
adjust patch T207916 |
[production] |
| 07:16 |
<bawolff@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T207916 13b993ab9f - auth log on in arwiki (duration: 00m 54s) |
[production] |
| 06:51 |
<bawolff@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: T207916 13b993ab9f - auth log on in group1 (duration: 00m 54s) |
[production] |
| 06:33 |
<moritzm> |
uploaded openjdk-8 backport for recent Java 8 security updates to apt.wikimedia.org/jessie-wikimedia |
[production] |
| 06:24 |
<bawolff@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: ecf579e9f9 - T207916 - enable auth log group0 (duration: 00m 55s) |
[production] |
| 06:18 |
<elukey> |
add AAAA DNS records for aqs and matomo1001 |
[analytics] |
| 06:01 |
<bawolff> |
adjust patch for T207916 |
[production] |
| 05:55 |
<elukey> |
reportupdater hadoop migrated to stat1007 |
[analytics] |
| 05:07 |
<bawolff@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: d3b2c346 T207916 (duration: 00m 55s) |
[production] |