9601-9650 of 10000 results (45ms)
2013-12-02 §
22:05 <catrope> synchronized wmf-config/InitialiseSettings.php 'Only activate VisualEditor in the User namespace on svwiktionary' [production]
22:04 <catrope> synchronized visualeditor-default.dblist 'Enable VisualEditor by default on 102 wikis' [production]
21:33 <RobH> ignore any alerts for testsearch1XXX, I just decommissioned them, but icinga hasn't updated quite yet [production]
21:02 <yurik> synchronized docroot and w [production]
20:46 <maxsem> synchronized php-1.23wmf5/extensions/MobileFrontend/ 'https://gerrit.wikimedia.org/r/#/c/95636/' [production]
20:45 <maxsem> synchronized php-1.23wmf4/extensions/MobileFrontend/ 'https://gerrit.wikimedia.org/r/#/c/95636/' [production]
20:10 <yurik> synchronized w/mobilelanding.php [production]
20:02 <yurik> synchronized php-1.23wmf5/extensions/ZeroRatedMobileAccess/ [production]
19:53 <yurik> synchronized php-1.23wmf4/extensions/ZeroRatedMobileAccess/ [production]
19:01 <demon> synchronized php-1.23wmf4/extensions/Elastica 'Fix missing code in Elastica on 1.23wmf4 wikis' [production]
18:58 <K4-713> updated fraud rules on payments [production]
18:53 <demon> synchronized wmf-config/CommonSettings.php 'Search updates back off for Cirrus wikis :(' [production]
18:35 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Dec 2 18:35:42 UTC 2013 [production]
18:34 <ottomata> added logster deb, installed on cp3003 for testing, will puppetize shortly [production]
18:30 <demon> synchronized wmf-config/CommonSettings.php 'Search updates back on for Cirrus' [production]
17:47 <LocalisationUpdate> completed (1.23wmf5) at Mon Dec 2 17:47:29 UTC 2013 [production]
17:31 <bblack> cp301[12].esams - puppet temporarily disabled, custom crash handler vmod in place to try to catch an error in the next couple of hours [production]
17:31 <demon> synchronized wmf-config/CommonSettings.php 'Search update off for cirrus wikis' [production]
17:31 <LocalisationUpdate> completed (1.23wmf4) at Mon Dec 2 17:31:11 UTC 2013 [production]
17:30 <demon> updated /a/common to {{Gerrit|I507a72cca}}: Fix submodule reference change that snuck into If5b3a27a [production]
17:21 <demon> synchronized wmf-config/InitialiseSettings.php 'Cirrus on all the wikis (that had it before)' [production]
17:09 <demon> synchronized php-1.23wmf5/extensions/CirrusSearch 'Cirrus to master' [production]
17:07 <demon> synchronized php-1.23wmf4/extensions/CirrusSearch 'Cirrus to master' [production]
17:06 <demon> synchronized wmf-config/CirrusSearch-common.php [production]
16:33 <paravoid> rebooting cp1065, usual XFS deadlock [production]
12:35 <Jeff_Green> stopped mysql on db1008 to clone a database [production]
07:05 <springle> upgrade/reboot db1046 m2 slave [production]
06:53 <springle> synchronized wmf-config/db-eqiad.php 'repool pc1003 after upgrade, max_connections lowered during warm up' [production]
06:28 <paravoid> fixed ganglia for misc eqiad (possibly others); see {{Gerrit|Icc5376505}} [production]
06:04 <springle> synchronized wmf-config/db-eqiad.php 'depool pc1003 for upgrade' [production]
02:51 <springle> synchronized wmf-config/db-eqiad.php 'repool pc1002 after upgrade, max_connections lowered during warm up' [production]
02:37 <LocalisationUpdate> ResourceLoader cache refresh completed at Mon Dec 2 02:37:48 UTC 2013 [production]
02:23 <springle> synchronized wmf-config/db-eqiad.php 'depool pc1002 for upgrade' [production]
02:15 <LocalisationUpdate> completed (1.23wmf5) at Mon Dec 2 02:14:57 UTC 2013 [production]
02:08 <LocalisationUpdate> completed (1.23wmf4) at Mon Dec 2 02:08:15 UTC 2013 [production]
01:16 <springle> restarting labsdb1002 mysqld processes with 25% smaller buffer pools. kernel OOM killer striking. needs investigation [production]
2013-12-01 §
06:23 <apergos> sq37 hardware errors, probably controller, rt #6418 [production]
06:04 <apergos> pwercycled sq37, was giving 'i/o error' to all commands [production]
02:36 <LocalisationUpdate> ResourceLoader cache refresh completed at Sun Dec 1 02:36:06 UTC 2013 [production]
02:13 <LocalisationUpdate> completed (1.23wmf5) at Sun Dec 1 02:13:36 UTC 2013 [production]
02:08 <LocalisationUpdate> completed (1.23wmf4) at Sun Dec 1 02:07:54 UTC 2013 [production]
2013-11-30 §
05:54 <paravoid> rebooting cp1052 with the same kernel (control) [production]
02:47 <LocalisationUpdate> ResourceLoader cache refresh completed at Sat Nov 30 02:47:22 UTC 2013 [production]
02:18 <LocalisationUpdate> completed (1.23wmf5) at Sat Nov 30 02:18:36 UTC 2013 [production]
02:10 <LocalisationUpdate> completed (1.23wmf4) at Sat Nov 30 02:10:13 UTC 2013 [production]
2013-11-29 §
21:09 <ori> synchronized wmf-config/InitialiseSettings.php 'Id9c7321b8: Add a MassMessage-related user group on Meta' [production]
21:08 <ori> updated /a/common to {{Gerrit|Id9c7321b8}}: Add a MassMessage-related user group on Meta [production]
20:18 <paravoid> rebooting cp1065 with new kernel [production]
19:26 <paravoid> "swapoff -a" on all cache_text to deal with strange kernel issue with kswapd dropping the whole page cache on memory pressure [production]
14:49 <paravoid> restarted gmond on ms-fe1001/2, both were stuck 6h ago and we lost all swift eqiad's metrics for that period [production]