3651-3700 of 10000 results (48ms)
2017-08-15 §
13:21 <moritzm> bounced pdfrender on scb1001 (T159922) [production]
11:32 <moritzm> installing Linux updates on stretch systems (no reboots yet) [production]
10:45 <moritzm> installing PHP security updates on trusty [production]
10:09 <jynus> rebooting db1078 [production]
09:39 <jynus@tin> Synchronized wmf-config/db-eqiad.php: depool db1078 (duration: 00m 49s) [production]
07:56 <moritzm> installing cvs security updates [production]
07:18 <elukey> restart pdfrender on scb1003 [production]
02:56 <l10nupdate@tin> ResourceLoader cache refresh completed at Tue Aug 15 02:56:14 UTC 2017 (duration 6m 42s) [production]
02:49 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.13) (duration: 07m 53s) [production]
02:28 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.11) (duration: 08m 58s) [production]
2017-08-14 §
22:16 <Dereckson> Previous log entry is related to [[Gerrit:371960]] and [[Gerrit:371961]]. [production]
22:15 <dereckson@tin> Synchronized php-1.30.0-wmf.11/extensions/EventBus/JobQueueEventBus.php: JobQueueEventBus: Populate the database field + not set properties are accessed (duration: 00m 48s) [production]
20:44 <ppchelko@tin> Finished deploy [restbase/deploy@4d6c706]: Temporary fallback to the new storage buckets before truncation (duration: 09m 35s) [production]
20:35 <ppchelko@tin> Started deploy [restbase/deploy@4d6c706]: Temporary fallback to the new storage buckets before truncation [production]
17:09 <gehel> resetting mode on stat1005:/srv/published-datasets/discovery recursively [production]
16:29 <elukey> execute sudo find -type f -mtime +60 -exec rm {} \; in /var/lib/carbon on graphite2001 to free some space in / [production]
13:32 <elukey> Execute systemctl mask nfacctd on rhenium.wikimedia.org for T172681 [production]
13:08 <zeljkof> nothing for EU SWAT [production]
12:21 <jynus> stopping replication on all instances of dbstore2001 T169516 [production]
11:01 <marostegui@tin> Synchronized wmf-config/db-eqiad.php: Add db2076 - T170662 T151029 (duration: 00m 47s) [production]
11:00 <marostegui@tin> Synchronized wmf-config/db-codfw.php: Add db2076 - T170662 T151029 (duration: 01m 02s) [production]
10:34 <marostegui> Restart MySQL on db1069 to pick up new replcation filters [production]
10:27 <marostegui> Restart MySQL on db1095 to pick up new replication filters [production]
09:16 <oblivian@puppetmaster1001> conftool action : set/pooled=yes; selector: service=thumbor,name=thumbor2001.codfw.wmnet [production]
09:07 <_joe_> stopping thumbor on thumbor2001 after depooling it for testing [production]
09:05 <oblivian@puppetmaster1001> conftool action : set/pooled=no; selector: service=thumbor,name=thumbor2001.codfw.wmnet [production]
03:15 <l10nupdate@tin> ResourceLoader cache refresh completed at Mon Aug 14 03:15:31 UTC 2017 (duration 7m 6s) [production]
03:08 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.13) (duration: 06m 47s) [production]
02:39 <l10nupdate@tin> scap sync-l10n completed (1.30.0-wmf.11) (duration: 08m 29s) [production]
2017-08-13 §
22:45 <ebernhardson> restart elsaticsearch on elastic1017 after setting md2 readahead to 256 to match md2 on 1032-152 [production]
19:35 <godog> upload python-thumbor-wikimedia 1.2 - T161719 [production]
19:01 <godog> bounce pdfrender on scb1001 and scb1003 - T159922 [production]
18:46 <godog> bounce carbon and uwsgi on graphite1003 [production]
17:53 <reedy@tin> Synchronized wmf-config/db-eqiad.php: fix comment (duration: 00m 47s) [production]
17:52 <reedy@tin> Synchronized wmf-config/db-codfw.php: fix comment (duration: 00m 47s) [production]
17:51 <reedy@tin> Synchronized refresh-dblist: phpcs (duration: 00m 48s) [production]
2017-08-12 §
20:00 <krinkle@tin> Synchronized php-1.30.0-wmf.13/includes/jobqueue/JobQueueGroup.php: T171371 - Log job pushes to bogus wikis (duration: 00m 53s) [production]
16:09 <Reedy> Deleted some bogus user languages from commonswiki.user_properties [production]
15:25 <elukey> powercycle mw2256 (able to use com2 but not to login as root, regular ssh hanging) - T163346 [production]
2017-08-11 §
21:47 <legoktm@tin> Finished scap: (no justification provided) (duration: 07m 09s) [production]
21:41 <jmm@puppetmaster1001> conftool action : set/pooled=inactive; selector: mw2256.codfw.wmnet [production]
21:40 <legoktm@tin> Started scap: (no justification provided) [production]
21:40 <legoktm@tin> scap failed: LockFailedError Failed to acquire lock "/var/lock/scap.unknown-but-probably-mediawiki.lock"; owner is "legoktm"; reason is "Deploying Timeless (try 2) - T154371" (duration: 00m 00s) [production]
21:02 <ebernhardson> unban elastic1017 from elasticsearch cluster [production]
20:52 <bblack> varnish backend restart on cp1099 [production]
20:46 <legoktm@tin> Started scap: Deploying Timeless (try 2) - T154371 [production]
20:42 <legoktm@tin> scap aborted: Deploying Timeless - T154371 (duration: 05m 10s) [production]
20:36 <legoktm@tin> Started scap: Deploying Timeless - T154371 [production]
20:19 <bblack> varnish backend restart on cp1049 + cp1074 (mailbox lag) [production]
19:59 <ebernhardson> ban elastic1017 from eqiad search cluster [production]