1551-1600 of 10000 results (28ms)
2015-11-30 §
02:00 <l10nupdate@tin> LocalisationUpdate failed: git pull of core failed [production]
01:56 <gwicke> started `nodetool cleanup` on restbase1002 to get rid of unnecessary data from earlier 1001 decommission attempt [production]
01:05 <bd808@tin> sync-l10n completed (1.27.0-wmf.7) (duration: 01m 19s) [production]
01:04 <bd808> testing l10n cache rebuild as l10nupdate user (take 2) [production]
00:57 <Krenair> test [production]
00:49 <bd808@tin> sync-l10nupdate completed (1.27.0-wmf.7) (duration: 04m 37s) [production]
00:45 <bd808> testing l10n cache rebuild as l10nupdate user [production]
00:01 <bd808> Tried to update scap to 1879fd4 (Add sync-l10n command for l10nupdate); trebuchet reported 0/483 minions completing fetch and 3/483 minions completing checkout [production]
2015-11-29 §
23:23 <bd808> updated cherry-pick of https://gerrit.wikimedia.org/r/#/c/255916/ to PS2 [releng]
21:25 <jynus> importing user.user_touched (s7) from dbstore1002 to sanitarium. s7 lag on labs replicas will be higher for some minutes. [production]
20:51 <jynus> importing user.user_touched (s6) from dbstore1002 to sanitarium. s6 lag on labs replicas will be higher for some minutes. [production]
20:28 <jynus> importing user.user_touched (s5) from dbstore1002 to sanitarium. s5 lag on labs replicas will be higher for some minutes. [production]
19:51 <jynus> importing user.user_touched (s4) from dbstore1002 to sanitarium. s4 lab will be affected for some minutes. [production]
05:34 <bd808> cherry-picked https://gerrit.wikimedia.org/r/#/c/255916/ for testing [releng]
05:25 <bd808> trebuchet is wack and not getting returner results from any hosts; see T119765 [releng]
05:23 <bd808> updated scap to 1879fd4 Add sync-l10n command for l10nupdate [releng]
05:22 <bd808> stashed uncommited scap3 changes found on deployment-bastion [releng]
04:50 <gwicke> restarted cassandra on restbase1009 to avoid it running out of disk space; had large compaction (~2TB) at 80% and only 64G disk space left [production]
03:01 <YuviPanda> run chown -R l10nupdate: /var/lib/l10nupdate/mediawiki for Reedy on tin [production]
02:28 <Reedy> l10nupdate failed because some git objects owned by 997:l10nupdate [production]
02:00 <l10nupdate@tin> LocalisationUpdate failed: git pull of core failed [production]
2015-11-28 §
22:48 <bd808@tin> Synchronized php-1.27.0-wmf.5/cache/l10n: bd808 testing l10nupdate sync-dir using stale branch (duration: 01m 29s) [production]
20:49 <l10nupdate@tin> LocalisationUpdate failed: Failed to sync-dir 'php-1.27.0-wmf.7/cache/l10n' [production]
20:49 <krenair@tin> Synchronized php-1.27.0-wmf.7/cache/l10n: l10nupdate for 1.27.0-wmf.7 (duration: 07m 11s) [production]
20:35 <ori@tin> Synchronized wmf-config/InitialiseSettings.php: Ie33ae3b6a: Increase $wgCopyUploadTimeout to 90 seconds (from default 25) (T118887) (duration: 00m 27s) [production]
19:35 <rxy> Setting up Cubbie in cvn-app5. [cvn]
2015-11-26 §
07:15 <ori> rdb100x migrated to redis::instance, meaning they each have two additional redis instances, on port 6380 and 6381, accordingly. [production]
06:45 <ori@tin> Synchronized wmf-config/jobqueue-eqiad.php: I3e6021ed3: Update job queue config to use 2 new rdb1003 instances (duration: 00m 28s) [production]
02:00 <l10nupdate@tin> LocalisationUpdate failed: git pull of core failed [production]
00:14 <ori@tin> Synchronized wmf-config/jobqueue-eqiad.php: I1e0d0ed8215f: pool rdb1007:6381 (duration: 00m 30s) [production]
2015-11-25 §
23:21 <krenair@tin> Synchronized private/README_BEFORE_MODIFYING_ANYTHING: 334ca105e92aaf7046e244ff39189f3823d31a7d (duration: 00m 32s) [production]
22:14 <demon@tin> Finished scap: new MW release, swapping extdist config + msgs (duration: 23m 59s) [production]
21:50 <demon@tin> Started scap: new MW release, swapping extdist config + msgs [production]
20:31 <gwicke> running `nodetool cleanup` on restbase1007 to make sure that we don't have extra sstables from the 1001 decommision taking up space [production]
19:53 <legoktm> ran mwscript sql.php --wiki=enwiki --wikidb=wikishared /srv/mediawiki-staging/php-master/extensions/Echo/db_patches/echo_unread_wikis.sql [releng]
19:13 <mobrovac> restbase deploy end of 74662c [production]
19:12 <moritzm> installed django security updates on stat* and graphite hosts [production]
18:47 <robh> removing cr2-ulsfo:xe-1/2/0, Patch ID 1062 as T118171 cancels that link [production]
18:46 <mobrovac> restbase deploy start of 74662c [production]
18:19 <bd808@tin> Synchronized README: Testing l10nupdate uid fix for T119165 (duration: 00m 28s) [production]
18:04 <andrewbogott> restored pdns-recursor on holmium, again [production]
18:00 <mobrovac> restbase canary deploy to restbase1001 of 74662c6 [production]
17:44 <andrewbogott> killing pdns-recursor on holmium [production]
16:53 <andrewbogott> killing pdns-recursor on holmium [production]
16:16 <jynus@tin> Synchronized wmf-config/db-eqiad.php: Repool db1044 after maintenance (duration: 00m 47s) [production]
15:53 <godog> ban grafana kafka dashboard temporarily from graphite [production]
15:42 <Coren> migrating tools-web-static-02 to labvirt1010 to free space on labvirt1002 [tools]
14:57 <godog> bounce uwsgi on graphite1001 [production]
14:52 <godog> stop puppet on restbase1* / restbase2* before https://gerrit.wikimedia.org/r/#/c/254372/ [production]
13:09 <hashar@tin> Synchronized php-1.27.0-wmf.7/Rakefile: Added Rakefile https://gerrit.wikimedia.org/r/#/c/254423/ (duration: 00m 28s) [production]