|
2017-11-14
§
|
| 12:57 |
<godog> |
upgrade grafana to 4.6.1 on https://grafana-labs.wikimedia.org/ - T180428 |
[production] |
| 12:03 |
<ema> |
restart varnish-be on cp3007, requests failing with 'no backend connection' |
[production] |
| 11:49 |
<moritzm> |
rebooting remaining app servers in eqiad for update to Linux 4.9.51 (and to pick up OpenSSL updates) |
[production] |
| 11:22 |
<moritzm> |
rebooting remaining API servers in eqiad for update to Linux 4.9.51 (and to pick up OpenSSL updates) |
[production] |
| 11:04 |
<godog> |
reimage restbase2002 - T179422 |
[production] |
| 10:50 |
<marostegui> |
Deploy alter table on s5: db1104 db1100 db1106 - T174569 |
[production] |
| 10:32 |
<elukey> |
removed old target configs from /srv/prometheus/analytics/targets on prometheus100[34] after https://gerrit.wikimedia.org/r/391179 |
[production] |
| 10:23 |
<moritzm> |
rebooting image scalers in eqiad for update to Linux 4.9.51 (and to pick up OpenSSL updates) |
[production] |
| 10:07 |
<godog> |
upload scap 3.7.2-1 - T127762 |
[production] |
| 09:59 |
<mobrovac@tin> |
Synchronized wmf-config/jobqueue.php: JobQueue: Migrate RecordLintJob to EventBus - T175212 (duration: 00m 46s) |
[production] |
| 09:56 |
<marostegui> |
Deploy alter table on s5 - dbstore1001 - T174569 |
[production] |
| 09:49 |
<foks> |
Disabled 2FA for Jean-Frédéric |
[production] |
| 09:46 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Pool db1105 as multi-instance host for s1 and s2 - T178359 (duration: 00m 46s) |
[production] |
| 09:45 |
<marostegui@tin> |
Synchronized wmf-config/db-codfw.php: Pool db1105 as multi-instance host for s1 and s2 - T178359 (duration: 00m 47s) |
[production] |
| 09:43 |
<godog> |
upgrade prometheus to 1.8.1 with k8s on prometheus2004 - T177395 |
[production] |
| 09:27 |
<ppchelko@tin> |
Finished deploy [cpjobqueue/deploy@df5fca9]: Enable RecordLintJob processing (duration: 00m 40s) |
[production] |
| 09:27 |
<ppchelko@tin> |
Started deploy [cpjobqueue/deploy@df5fca9]: Enable RecordLintJob processing |
[production] |
| 09:07 |
<moritzm> |
rebooting remaining app servers in codfw for update to Linux 4.9.51 (and to pick up OpenSSL updates) |
[production] |
| 09:05 |
<jmm@puppetmaster1001> |
conftool action : set/pooled=yes; selector: mw2108.codfw.wmnet |
[production] |
| 09:05 |
<jmm@puppetmaster1001> |
conftool action : set/pooled=yes; selector: wtp2018.codfw.wmnet |
[production] |
| 08:31 |
<marostegui> |
Deploy alter table on s5 - dbstore1002 - T174569 |
[production] |
| 08:22 |
<addshore@tin> |
Synchronized wmf-config/extension-list-labs: [[gerrit:390385|Add AdvancedSearch to extension-list]] T180147 PT 2/2 LABS ONLY (duration: 00m 46s) |
[production] |
| 08:21 |
<addshore@tin> |
Synchronized wmf-config/extension-list: [[gerrit:390385|Add AdvancedSearch to extension-list]] T180147 PT 1/2 (duration: 00m 47s) |
[production] |
| 08:13 |
<moritzm> |
rebooting job runners in codfw for update to Linux 4.9.51 (and to pick up OpenSSL updates) |
[production] |
| 06:28 |
<marostegui@tin> |
Synchronized wmf-config/db-eqiad.php: Increase weight for db1103 on s2 and s4 - T178359 (duration: 00m 47s) |
[production] |
| 06:18 |
<marostegui> |
Deploy alter table on s6 primary master (db1061) - T174569 |
[production] |
| 02:30 |
<l10nupdate@tin> |
ResourceLoader cache refresh completed at Tue Nov 14 02:30:59 UTC 2017 (duration 6m 38s) |
[production] |
| 02:24 |
<l10nupdate@tin> |
scap sync-l10n completed (1.31.0-wmf.7) (duration: 07m 28s) |
[production] |
| 01:35 |
<mdholloway> |
mobileapps finished rolling back to 11/8 deployment |
[production] |
| 01:34 |
<mholloway-shell@tin> |
Finished deploy [mobileapps/deploy@00e60b2]: (no justification provided) (duration: 02m 27s) |
[production] |
| 01:31 |
<mholloway-shell@tin> |
Started deploy [mobileapps/deploy@00e60b2]: (no justification provided) |
[production] |
| 01:30 |
<mdholloway> |
mobileapps rolling back today's deployment due to a significant increase in errors. |
[production] |
|
2017-11-13
§
|
| 22:49 |
<bawolff> |
deployed patch T119158 (Will affect language converter. -{}- no longer allowed in link urls) |
[production] |
| 22:42 |
<bawolff> |
deployed patch T124404 |
[production] |
| 22:02 |
<mholloway-shell@tin> |
Finished deploy [mobileapps/deploy@9b10959]: Update mobileapps to c002862 (duration: 05m 31s) |
[production] |
| 21:57 |
<mholloway-shell@tin> |
Started deploy [mobileapps/deploy@9b10959]: Update mobileapps to c002862 |
[production] |
| 21:40 |
<urandom> |
Decommissioning Cassandra, restbase1012-b.eqiad.wmnet (T179422) |
[production] |
| 20:20 |
<ejegg> |
updated CiviCRM from ddc388195e1974e17c1f6f3f67f285898c0eb2bb to f571c67ceb353b457f250cc4606cfaa882343e39 |
[production] |
| 19:20 |
<smalyshev@tin> |
Finished deploy [wdqs/wdqs@b44cf27]: data reload/T176593 (duration: 00m 20s) |
[production] |
| 19:20 |
<smalyshev@tin> |
Started deploy [wdqs/wdqs@b44cf27]: data reload/T176593 |
[production] |
| 19:18 |
<thcipriani@tin> |
Synchronized wmf-config/abusefilter.php: SWAT: [[gerrit:390153|Enable per-filter profiling on enwiki]] T179323 (duration: 00m 45s) |
[production] |
| 19:16 |
<thcipriani@tin> |
Synchronized docroot/search.wikimedia.org/index.php: [[gerrit:390357|search.wikimedia.org: Clean up result returning logic]] (duration: 00m 47s) |
[production] |
| 19:15 |
<madhuvishy> |
Kick of second dumps rsync from ms1001 to labstore1006 |
[production] |
| 18:28 |
<gehel> |
deploy latest blazegraph + GUI on wdqs200[23] to switch vocabulary - T176593 |
[production] |
| 18:20 |
<elukey> |
drain + shutdown analytics1029 as prep step to replace the BBU - T178742 |
[production] |
| 18:09 |
<hoo> |
Ran "scap pull" on mwdebug1001/snapshot1001 after (further) tests re T177486 |
[production] |
| 18:02 |
<urandom> |
Restarting Cassandra, restbase-dev1004.eqiad.wmnet (testing new `c-foreach-restart`) |
[production] |
| 18:00 |
<demon@tin> |
Synchronized docroot/search.wikimedia.org/index.php: minor cleanup (duration: 00m 47s) |
[production] |
| 17:28 |
<hoo> |
Ran "scap pull" on mwdebug1001 after tests re T177486 |
[production] |
| 17:12 |
<mobrovac> |
restbase depooling restbase1007, restbase1012, restbase1014, restbase2002, restbase2004, restbase2006 for T179422 |
[production] |