|
2017-07-28
§
|
| 07:52 |
<gehel> |
repooling wdqs1001 (data import completed) |
[production] |
| 07:52 |
<elukey> |
forced mii-tool -r eth0 on analytics1034 to get 1G negotiated speed |
[production] |
| 07:52 |
<gehel@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=wdqs1001.eqiad.wmnet |
[production] |
| 06:35 |
<moritzm> |
installing apache security updates on trusty systems |
[production] |
| 02:26 |
<mutante> |
scb1002 - systemctl restart pdfrender - was "connect to address 10.64.16.21 and port 5252: Connection refused" in Icinga since a couple hours (T159922) - recovered |
[production] |
| 02:08 |
<ottomata> |
stat1002: disabled puppet, umounted /tmp, /home and /a, poweroff |
[analytics] |
| 02:08 |
<ottomata> |
stat1002: disabled puppet, umounted /tmp, /home and /a, poweroff |
[production] |
| 00:51 |
<mutante> |
releases1001 - rsynced reprepro db data from bromine |
[production] |
| 00:27 |
<mutante> |
bromine sudo -E reprepro clearvanished to deleted unused precise-mediawiki causing reprepro errors |
[production] |
| 00:17 |
<Krinkle> |
Testing job insertion on beta cluster from deployment-tin triggers PHP Notice: Undefined index: uuid in EventBus/JobQueueEventBus.php:102, PHP Notice: Undefined index: sha1 in EventBus/JobQueueEventBus.php:99 |
[releng] |
|
2017-07-27
§
|
| 23:48 |
<catrope@tin> |
Synchronized php-1.30.0-wmf.11/resources/src/mediawiki.rcfilters/: T171368 (duration: 00m 42s) |
[production] |
| 23:39 |
<eileen> |
update process-control to 24c7bbe699a6bb685 (renable omnirecipient) |
[production] |
| 23:36 |
<eileen> |
update process-control to 2c1c8a3bcb0186 - new frequency on receipient load |
[production] |
| 23:35 |
<catrope@tin> |
Synchronized wmf-config/: Enable emails for minor edits everywhere but keep default prefs (T29884, T142727) (duration: 00m 45s) |
[production] |
| 23:27 |
<bd808> |
Killed python procs owned by sdesabbata on tools-login that were stealing all cpu/io |
[tools] |
| 23:17 |
<bd808> |
Re-enabled hourly cron after patching for T168653 with 3f1ea69 |
[tools.grid-jobs] |
| 23:16 |
<ejegg|away> |
disabled Omnimail recipient load job |
[production] |
| 23:14 |
<catrope@tin> |
Finished scap: wmf-config/InitialiseSettings.php Enable experimental RCFilters on group2 too (duration: 02m 55s) |
[production] |
| 23:11 |
<catrope@tin> |
Started scap: wmf-config/InitialiseSettings.php Enable experimental RCFilters on group2 too |
[production] |
| 22:46 |
<ejegg> |
enabled omnimail recipient load job, throttling inserts to 15,000 every 60 sec |
[production] |
| 22:27 |
<bd808> |
tool-admin-web is live. Switch back with ~/old.sh if things go sideways. (T140254) |
[tools.admin] |
| 22:22 |
<bd808> |
Attempting to switch webservice to tool-admin-web |
[tools.admin] |
| 22:21 |
<bd808> |
Updated tool-admin-web to 3830db8 |
[tools.admin] |
| 21:46 |
<ejegg> |
updated CiviCRM from ceff739dcfeb5e5bcf40d880146cfb44eaf462ea to 23f2bbf73557a7a88e783f68459112cf4bba1c79 |
[production] |
| 21:40 |
<urandom> |
Restarting Cassandra, restbase-dev1004-{a,b} to apply updated data directories list |
[production] |
| 21:16 |
<bd808> |
Disabled puppet on tools-proxy-01 to test nginx proxy config changes |
[tools] |
| 21:08 |
<herron> |
bast3002 repointed mdadm at null alias to clear systemd degraded state alert |
[production] |
| 20:31 |
<gwicke> |
restarting all pdfrender instances on scb in eqiad; one of them was hanging & causing user requests to fail |
[production] |
| 20:24 |
<Krinkle> |
Un-dirtying state of /srv/deployment/jobrunner/jobrunner on tin (from T129148). Checking-out https://gerrit.wikimedia.org/r/367743 instead. |
[production] |
| 20:22 |
<moritzm> |
installing apache security updates on cobalt |
[production] |
| 20:13 |
<ppchelko@tin> |
Finished deploy [restbase/deploy@cfb9c46]: Correctly escape the quotes in PDF names (duration: 07m 48s) |
[production] |
| 20:05 |
<ppchelko@tin> |
Started deploy [restbase/deploy@cfb9c46]: Correctly escape the quotes in PDF names |
[production] |
| 19:50 |
<thcipriani@tin> |
rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.30.0-wmf.11 |
[production] |
| 19:43 |
<mutante> |
switching https://releases.wikimedia.org backend from bromine to releases1001 - all files have been rsynced (T164030) |
[production] |
| 19:37 |
<robh> |
cp4021 shutting down for relocation in rack, will put in maint mode for next 2 hours |
[production] |
| 19:21 |
<catrope@tin> |
Synchronized wmf-config: T171556 (duration: 00m 47s) |
[production] |
| 19:20 |
<zhuyifei1999_> |
Updating to a6c5276 |
[tools.fontcdn] |
| 19:20 |
<catrope@tin> |
Synchronized dblists: T171556 (duration: 00m 46s) |
[production] |
| 19:11 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable namespace and tag filters in RCFilters on group0 and group1 (duration: 00m 46s) |
[production] |
| 19:07 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: T171751 (duration: 00m 46s) |
[production] |
| 19:06 |
<catrope@tin> |
Synchronized php-1.30.0-wmf.11/autoload.php: (no justification provided) (duration: 00m 45s) |
[production] |
| 19:05 |
<catrope@tin> |
Synchronized php-1.30.0-wmf.11/includes/: T168501, take two (duration: 01m 27s) |
[production] |
| 19:05 |
<mutante> |
added new misc::cache director "releases" for releases* servers, releases moving away from bromine (T164030) |
[production] |
| 18:48 |
<zhuyifei1999_> |
Updating to 022bb0a |
[tools.fontcdn] |
| 18:44 |
<catrope@tin> |
Synchronized php-1.30.0-wmf.11/includes/: Temporarily revert patch for T168501 while I fix it (duration: 01m 27s) |
[production] |
| 18:34 |
<zhuyifei1999_> |
attempting to update to e900070 (weird git hash, I know) |
[tools.fontcdn] |
| 18:32 |
<mutante> |
labpuppetmaster1001 - restarted ferm twice, DNS lookup for AAAA worked, error gone on second time. then did same on labpuppetmaster1002 (T171880) |
[production] |
| 18:29 |
<catrope@tin> |
Synchronized php-1.30.0-wmf.11/includes/: T168501 and T163380 (duration: 01m 31s) |
[production] |
| 18:22 |
<catrope@tin> |
Synchronized wmf-config/InitialiseSettings.php: Enable WikidataPageBanner on euwiki (T171763) (duration: 00m 46s) |
[production] |
| 18:17 |
<TabbyCat> |
Merge "Update composer.json" |
[tools.stewardbots] |