|
2018-05-01
§
|
| 21:23 |
<urandom> |
restbase: begin culling leaked revisions, enwiki_T_mobile__ng_{lead,remaining} - T192689 |
[production] |
| 20:43 |
<urandom> |
restbase: begin culling leaked revisions, commons_T_mobile__ng_remaining - T192689 |
[production] |
| 20:35 |
<demon@tin> |
Started scap: group0 to wmf.2 |
[production] |
| 20:35 |
<ebernhar|lunch> |
update stale apifeatureusage-search-svc-eqiad-wmnet template in eqiad elasticsearch and delete unused apifeatureusage template |
[production] |
| 20:28 |
<urandom> |
restbase: begin culling leaked revisions, commons_T_mobile__ng_lead - T192689 |
[production] |
| 20:06 |
<demon@tin> |
Pruned MediaWiki: 1.31.0-wmf.27 (duration: 03m 08s) |
[production] |
| 20:00 |
<ottomata> |
rolling restart of eventbus to apply new logstash formatter version T193230 |
[production] |
| 19:59 |
<demon@tin> |
Pruned MediaWiki: 1.31.0-wmf.30 [keeping static files] (duration: 01m 43s) |
[production] |
| 19:49 |
<urandom> |
restbase: begin culling leaked revisions, others_T_mobile__ng_remaining - T192689 |
[production] |
| 19:27 |
<ottomata> |
rolling restart of eventbus to apply logstash tag https://phabricator.wikimedia.org/T193230 |
[production] |
| 19:17 |
<mutante> |
mw2174,mw2175,mw2176 ff - reinstalling with wmf-auto-reimage to stretch |
[production] |
| 19:00 |
<ottomata> |
rolling restart of eventlogging-service-eventbus to apply logstash logging configs - T193230 |
[production] |
| 18:53 |
<ariel@tin> |
Finished deploy [dumps/dumps@5438d41]: keep running even if file we want to report on is moved/gone (duration: 00m 04s) |
[production] |
| 18:53 |
<ariel@tin> |
Started deploy [dumps/dumps@5438d41]: keep running even if file we want to report on is moved/gone |
[production] |
| 17:40 |
<XioNoX> |
disabling eqsin<->codfw link for high packet loss on link |
[production] |
| 17:39 |
<otto@tin> |
Finished deploy [eventlogging/eventbus@c70e8c5]: remove occasional logging of request.body in prep for T193230 (duration: 02m 29s) |
[production] |
| 17:36 |
<otto@tin> |
Started deploy [eventlogging/eventbus@c70e8c5]: remove occasional logging of request.body in prep for T193230 |
[production] |
| 16:51 |
<Hauskatze> |
maurelio@deployment-tin:~$ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=deploymentwiki --logwiki=deploymentwiki 'Spambot~BobFeliciano' 'BobFeliciano' |
[releng] |
| 16:15 |
<ebernhardson> |
T192972 change eqiad elasticsearch disk watermarks from 85/85 to 80/80 to match disk space alerts |
[production] |
| 16:15 |
<herron> |
manually kicked off mirror@sodium:~$ /usr/local/sbin/update-ubuntu-mirror to clear ubuntu mirror out of sync alert |
[production] |
| 15:50 |
<urandom> |
restbase: begin culling leaked revisions, others_T_mobile__ng_lead -- T192689 |
[production] |
| 15:37 |
<ejegg> |
updated SmashPig standalone from a4de12d415 to 99585c8084 |
[production] |
| 15:33 |
<elukey> |
restart historical on druid1003 - exceptions in the logs |
[analytics] |
| 15:22 |
<elukey> |
restart druid-historical on druid1002 - Caused by: java.lang.IllegalArgumentException: Could not resolve type id 'hdfs' into a subtype of |
[analytics] |
| 14:05 |
<andrewbogott> |
moving tools-webgrid-lighttpd-1406 to labvirt1016 (routine rebalancing) |
[tools] |
| 13:35 |
<herron> |
restarted hhvm on mw1233 |
[production] |
| 11:44 |
<joal> |
False positive only in webrequest-load-check_sequence_statistics-wf-upload-2018-5-1-6 |
[analytics] |
| 07:14 |
<joal> |
Rerun webrequest-druid-daily-wf-2018-4-30 |
[analytics] |
| 06:24 |
<elukey> |
roll restart of all middlemanagers on druid100[123] - realtime tasks piled up from hours |
[analytics] |
| 02:47 |
<l10nupdate@tin> |
scap sync-l10n completed (1.32.0-wmf.1) (duration: 07m 04s) |
[production] |
| 01:20 |
<ejegg> |
re-enabled fundraising jobs |
[production] |
| 01:01 |
<ejegg> |
updated CiviCRM from 47197006d5 to 46883844a3 |
[production] |
| 00:51 |
<eileen> |
email sent advising of minor interuption |
[production] |
| 00:50 |
<ejegg> |
disabled fundraising jobs for db update |
[production] |
| 00:21 |
<ebernhardson> |
increase cluster.routing.allocation.balance.threshold from 1.0 to 1.5 for eqiad elasticsearch cluster to reduce rebalancing agressiveness |
[production] |
|
2018-04-30
§
|
| 23:38 |
<catrope@tin> |
Synchronized php-1.32.0-wmf.1/extensions/VisualEditor/modules/ve-mw/init/ve.init.mw.DiffPage.init.js: T192755 (duration: 00m 59s) |
[production] |
| 23:36 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Set $wgKartographerUsePageLanguage to false everywhere (T192955) (duration: 00m 59s) |
[production] |
| 23:33 |
<catrope@tin> |
Synchronized php-1.32.0-wmf.1/extensions/AbuseFilter/includes/AbuseFilter.php: Fix notices when disallowing edits (duration: 00m 59s) |
[production] |
| 23:21 |
<catrope@tin> |
Synchronized wmf-config/: USe internal cluster for SPARQL services (T192942) (duration: 01m 02s) |
[production] |
| 23:14 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Config cleanup patches from SWAT (duration: 01m 00s) |
[production] |
| 23:05 |
<mutante> |
ores1001: rm -rf /srv/deployment/ores/venv/ (T193422) |
[production] |
| 23:04 |
<ottomata> |
blacklisting change-prop and job queue topics from main-eqiad -> analytics (eqiad) |
[analytics] |
| 22:55 |
<ottomata> |
bouncing kafka main-eqiad -> eqiad (analytics) mirror maker |
[analytics] |
| 21:46 |
<ebernhardson> |
T192972 increase eqiad elasticsearch disk watermarks from 75/80 to 85/85 |
[production] |
| 20:27 |
<arlolra> |
Updated Parsoid to 50b0588 (T186358, T191700, T192909) |
[production] |
| 20:22 |
<awight@tin> |
Started deploy [ores/deploy@bf182e2]: ORES: Include bot edits in precaching wikidata itemquality; T187927 |
[production] |
| 20:21 |
<arlolra@tin> |
Finished deploy [parsoid/deploy@d8d7b42]: Updating Parsoid to 50b0588 (duration: 09m 46s) |
[production] |
| 20:20 |
<awight@tin> |
Finished deploy [ores/deploy@5b27205]: Rollback ores1001 to master (duration: 02m 56s) |
[production] |
| 20:19 |
<bsitzmann@tin> |
Finished deploy [mobileapps/deploy@d3724d2]: Update mobileapps to cc00cae (T191869) (duration: 07m 32s) |
[production] |
| 20:17 |
<awight@tin> |
Started deploy [ores/deploy@5b27205]: Rollback ores1001 to master |
[production] |