7001-7050 of 10000 results (56ms)
2017-07-28 §
02:26 <mutante> scb1002 - systemctl restart pdfrender - was "connect to address 10.64.16.21 and port 5252: Connection refused" in Icinga since a couple hours (T159922) - recovered [production]
02:08 <ottomata> stat1002: disabled puppet, umounted /tmp, /home and /a, poweroff [production]
00:51 <mutante> releases1001 - rsynced reprepro db data from bromine [production]
00:27 <mutante> bromine sudo -E reprepro clearvanished to deleted unused precise-mediawiki causing reprepro errors [production]
2017-07-27 §
23:48 <catrope@tin> Synchronized php-1.30.0-wmf.11/resources/src/mediawiki.rcfilters/: T171368 (duration: 00m 42s) [production]
23:39 <eileen> update process-control to 24c7bbe699a6bb685 (renable omnirecipient) [production]
23:36 <eileen> update process-control to 2c1c8a3bcb0186 - new frequency on receipient load [production]
23:35 <catrope@tin> Synchronized wmf-config/: Enable emails for minor edits everywhere but keep default prefs (T29884, T142727) (duration: 00m 45s) [production]
23:16 <ejegg|away> disabled Omnimail recipient load job [production]
23:14 <catrope@tin> Finished scap: wmf-config/InitialiseSettings.php Enable experimental RCFilters on group2 too (duration: 02m 55s) [production]
23:11 <catrope@tin> Started scap: wmf-config/InitialiseSettings.php Enable experimental RCFilters on group2 too [production]
22:46 <ejegg> enabled omnimail recipient load job, throttling inserts to 15,000 every 60 sec [production]
21:46 <ejegg> updated CiviCRM from ceff739dcfeb5e5bcf40d880146cfb44eaf462ea to 23f2bbf73557a7a88e783f68459112cf4bba1c79 [production]
21:40 <urandom> Restarting Cassandra, restbase-dev1004-{a,b} to apply updated data directories list [production]
21:08 <herron> bast3002 repointed mdadm at null alias to clear systemd degraded state alert [production]
20:31 <gwicke> restarting all pdfrender instances on scb in eqiad; one of them was hanging & causing user requests to fail [production]
20:24 <Krinkle> Un-dirtying state of /srv/deployment/jobrunner/jobrunner on tin (from T129148). Checking-out https://gerrit.wikimedia.org/r/367743 instead. [production]
20:22 <moritzm> installing apache security updates on cobalt [production]
20:13 <ppchelko@tin> Finished deploy [restbase/deploy@cfb9c46]: Correctly escape the quotes in PDF names (duration: 07m 48s) [production]
20:05 <ppchelko@tin> Started deploy [restbase/deploy@cfb9c46]: Correctly escape the quotes in PDF names [production]
19:50 <thcipriani@tin> rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.30.0-wmf.11 [production]
19:43 <mutante> switching https://releases.wikimedia.org backend from bromine to releases1001 - all files have been rsynced (T164030) [production]
19:37 <robh> cp4021 shutting down for relocation in rack, will put in maint mode for next 2 hours [production]
19:21 <catrope@tin> Synchronized wmf-config: T171556 (duration: 00m 47s) [production]
19:20 <catrope@tin> Synchronized dblists: T171556 (duration: 00m 46s) [production]
19:11 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable namespace and tag filters in RCFilters on group0 and group1 (duration: 00m 46s) [production]
19:07 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: T171751 (duration: 00m 46s) [production]
19:06 <catrope@tin> Synchronized php-1.30.0-wmf.11/autoload.php: (no justification provided) (duration: 00m 45s) [production]
19:05 <catrope@tin> Synchronized php-1.30.0-wmf.11/includes/: T168501, take two (duration: 01m 27s) [production]
19:05 <mutante> added new misc::cache director "releases" for releases* servers, releases moving away from bromine (T164030) [production]
18:44 <catrope@tin> Synchronized php-1.30.0-wmf.11/includes/: Temporarily revert patch for T168501 while I fix it (duration: 01m 27s) [production]
18:32 <mutante> labpuppetmaster1001 - restarted ferm twice, DNS lookup for AAAA worked, error gone on second time. then did same on labpuppetmaster1002 (T171880) [production]
18:29 <catrope@tin> Synchronized php-1.30.0-wmf.11/includes/: T168501 and T163380 (duration: 01m 31s) [production]
18:22 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable WikidataPageBanner on euwiki (T171763) (duration: 00m 46s) [production]
18:15 <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Limit FeaturedFeed on dewiki to 7 days (T159664) (duration: 00m 47s) [production]
18:05 <herron> installed libmail-spf-perl on fermium to address spamassassin "module not installed: Mail::SPF ('require' failed)" error [production]
17:05 <ejegg> restarted recurring Ingenico charge job [production]
16:01 <ejegg> updated CiviCRM from e83c012581305012145eae45495e7e8ea6f4e249 to ceff739dcfeb5e5bcf40d880146cfb44eaf462ea [production]
15:44 <jynus> stopping mysql, upgrading and restarting labsdb1010 [production]
15:03 <moritzm> stopping jobrunner/jobchron on mw1260 to investigate a few failing ffmpeg2theora invocations (T145742) [production]
13:31 <ema> lvs1009, lvs1010: upgrade to pybal 1.13.10 (one-packet-scheduling) T104442 [production]
13:19 <moritzm> installing apache updates on mendelevium and terbium [production]
13:08 <hashar@tin> Synchronized wmf-config/InitialiseSettings.php: Enable mapframe for cswiki - T171588 (duration: 00m 47s) [production]
10:32 <moritzm> reimaging mw2246 to jessie (T145742) [production]
10:31 <jynus> shutting down and rebooting db2016 [production]
10:14 <_joe_> restarting puppetdb on nihal, T170740 [production]
10:06 <jynus@tin> Synchronized wmf-config/db-codfw.php: Promote db2048 as the new codfw-s1 master (duration: 00m 46s) [production]
10:05 <jynus> starting actual master failover s1-codfw db2016->db2048 [production]
09:48 <ema> pybal 1.13.10 (one-packet-scheduling) built and uploaded to apt.w.o T104442 [production]
09:23 <jynus> starting s1-codfw database topology changes [production]