- Relanding 61718.

I disabled the GPU watchdog in three new cases:
- If the OSMesa software renderer is in use. This will disable it on bots.
- When running on valgrind, whether on a bot or locally.
- In debug builds

I added a GPU process initialization time to the GPU info.

I moved the GPU initialization code outside the watchdog protection because it
can take a long time and trigger the watchdog.

I increased the timeout. I set up a field trial with different timeouts to see
the rate of failure for each period.

Original CL description:

I added a watchdog thread that intermitently checks the main thread can respond
to tasks posted on its message queue.

I fixed some bugs that prevented GGL from failing when the GPU channel was
lost.

Added a command line swith to disable the watchdog thread for debugging
purposes.

TEST=try, local testing of all features
BUG=none


git-svn-id: svn://svn.chromium.org/chrome/trunk/src@65461 0039d316-1c4b-4281-b951-d872f2087c98
diff --git a/chrome/gpu/gpu_watchdog_thread.h b/chrome/gpu/gpu_watchdog_thread.h
new file mode 100644
index 0000000..d6e1117
--- /dev/null
+++ b/chrome/gpu/gpu_watchdog_thread.h
@@ -0,0 +1,41 @@
+// Copyright (c) 2010 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef CHROME_GPU_GPU_WATCHDOG_THREAD_H_
+#define CHROME_GPU_GPU_WATCHDOG_THREAD_H_
+
+#include "base/ref_counted.h"
+#include "base/scoped_ptr.h"
+#include "base/task.h"
+#include "base/thread.h"
+
+// A thread that intermitently sends tasks to a group of watched message loops
+// and deliberately crashes if one of them does not respond after a timeout.
+class GpuWatchdogThread : public base::Thread,
+                          public base::RefCountedThreadSafe<GpuWatchdogThread> {
+ public:
+  GpuWatchdogThread(MessageLoop* watched_message_loop, int timeout);
+  virtual ~GpuWatchdogThread();
+
+ protected:
+  virtual void Init();
+  virtual void CleanUp();
+
+ private:
+  void OnAcknowledge();
+  void OnCheck();
+  void PostAcknowledge();
+  void OnExit();
+  void Disable();
+
+  MessageLoop* watched_message_loop_;
+  int timeout_;
+
+  typedef ScopedRunnableMethodFactory<GpuWatchdogThread> MethodFactory;
+  scoped_ptr<MethodFactory> method_factory_;
+
+  DISALLOW_COPY_AND_ASSIGN(GpuWatchdogThread);
+};
+
+#endif  // CHROME_GPU_GPU_WATCHDOG_THREAD_H_