951-1000 of 10000 results (36ms)
2015-11-23 §
21:15 <jynus> enforcing limits on running queries on all labsdb databases [production]
20:47 <jynus> restarting labsdb1002, hopfully for the last time [production]
17:54 <jynus> temporarily stopping labsdb1002 to solve replication issues [production]
17:32 <Coren> emergency hotfix on labstore1001 (apt-get remove nfsd-ldap) as it appears to not work in all situations. [production]
17:30 <ori> restarting hhvm on mw1122 after lock-up; backtrace saved as /tmp/hhvm.32016.bt. [production]
17:28 <thcipriani@tin> Finished scap: SWAT: Update CentralNotice [[gerrit:254865]] (duration: 26m 31s) [production]
17:02 <thcipriani@tin> Started scap: SWAT: Update CentralNotice [[gerrit:254865]] [production]
16:44 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: noindex user namespace on en.wikipedia.org [[gerrit:237330]] (duration: 00m 28s) [production]
16:43 <jynus> restarting mysqld on labsdb1003 again [production]
16:38 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: End first mobile QuickSurveys campaign [[gerrit:254831]] (duration: 00m 28s) [production]
16:32 <thcipriani@tin> Synchronized php-1.27.0-wmf.7/includes/specials/SpecialWatchlist.php: SWAT: Special:Watchlist: Add user preference to "Show last" options, fix float comparison [[gerrit:254880]] (duration: 00m 29s) [production]
16:10 <thcipriani@tin> Synchronized wmf-config/InitialiseSettings.php: SWAT: CX: Enable article-recommender-1 campaign as default [[gerrit:254117]] (duration: 00m 52s) [production]
15:56 <paravoid> rebooted lvs3003 back with (an updated) 3.19 [production]
15:53 <_joe_> live-resizing /var/lib/wdqs on wdqs1002 [production]
15:15 <Coren> NFS server on labstore1001 restarted with no issues. [production]
15:11 <Coren> Restarting NFS daemon on labstore1001 w/ the new LDAP shim. [production]
14:53 <paravoid> powercycling cp1053, unresponsive, blank console [production]
14:53 <bblack> depooling cp1053 [production]
14:50 <ottomata> removing erbium udp2log varnishncsa instances from all caches [production]
14:38 <paravoid> rebooting lvs3003 with new kernel [production]
13:18 <jynus> restarting mysql on labsdb1003, unresponsive [production]
11:05 <_joe_> stopping apache on mw2051 for testing [production]
08:58 <godog> restbase1001 nodetool decommission [production]
08:57 <mobrovac> mathoid deploying ff03d56 [production]
08:12 <moritzm> restarted gitblit on antimony [production]
02:19 <l10nupdate@tin> Synchronized php-1.27.0-wmf.7/cache/l10n: l10nupdate for 1.27.0-wmf.7 (duration: 05m 57s) [production]
01:26 <Jamesofur> end enwiki test election from cli to avoid confusion with real election [production]
2015-11-22 §
21:30 <Reedy> restarting stuck Jenkins [production]
02:16 <l10nupdate@tin> Synchronized php-1.27.0-wmf.7/cache/l10n: l10nupdate for 1.27.0-wmf.7 (duration: 05m 22s) [production]
2015-11-21 §
03:19 <ori> SecurePoll arbcomlist.php run for enwiki (requested by Jamesofur) complete. [production]
02:20 <l10nupdate@tin> Synchronized php-1.27.0-wmf.7/cache/l10n: l10nupdate for 1.27.0-wmf.7 (duration: 05m 18s) [production]
00:50 <ori> update-ubuntu-mirror on carbon completed successfully ; going to re-enable icinga-wm [production]
00:46 <ori> rampant puppet failures appear to be the result of a Hash Sum mismatch error during apt-get update; attempting to fix by running sudo -u mirror /usr/local/sbin/update-ubuntu-mirror on carbon. [production]
00:37 <ori> disabled puppet on neon for one hour and killed icinga-wm to silence a tide of puppet failure alerts [production]
00:24 <demon@tin> Synchronized .arcconfig: no-op, for completeness (duration: 00m 31s) [production]
2015-11-20 §
21:45 <csteipp@tin> Synchronized php-1.27.0-wmf.7/includes/User.php: Deploy fix for T119021 (duration: 00m 28s) [production]
18:55 <ori> resuming interrupted arbcomlist invocation on terbium [production]
16:51 <_joe_> decommissioning mw1041: removed facts and certs, removed from conftool and pybal [production]
16:44 <reedy@tin> Synchronized wmf-config/throttle.php: Add throttle config for Viquimarató nit digital/2015 (duration: 00m 28s) [production]
16:17 <_joe_> depooled mw1041 [production]
15:41 <Reedy> running sync-common on mw1041 [production]
15:32 <_joe_> powercycling mw1041 [production]
15:26 <reedy@tin> Synchronized wmf-config/throttle.php: Update throttle for event tomorrow (duration: 00m 28s) [production]
14:35 <godog> swift eqiad-prod: set ms-be1019 / ms-be1020 / ms-be1021 weight 3000 [production]
10:06 <ori> Running SecurePoll's arbcomlist.php on terbium with performance fix from https://gerrit.wikimedia.org/r/#/c/254375/ , per request from Jamesofur. Looks to be behaving well, can be killed if it acts up. [production]
07:44 <_joe_> rebooted pybal-test2001 [production]
07:36 <_joe_> powercycled mw1041 [production]
04:55 <twentyafterfour@tin> Synchronized multiversion/checkoutMediaWiki.php: inconsequential sync to test mirroring (duration: 02m 14s) [production]
04:41 <twentyafterfour@tin> Synchronized multiversion/checkoutMediaWiki.php: deploying I2ec91aa33e05c2dc367db8a6f6ba56be5397906b to test scap mirroring (duration: 00m 20s) [production]
04:36 <twentyafterfour> deploying scap on tin [production]