pkg/cdi: don't crash if we fail to create fsnotify watch.

Don't crash in update() if we fail to create an fsnotify watch.
This can happen if we have too many open files. In this case we
now record a failure for all configured spec directories and in
update we always trigger a refresh. If the process if ever able
to create new file descriptors the cache becomes functional but
in a 'always implicitly fully refreshed' mode instead of auto-
refreshed.

It's not entirely clear what is the best option to deal with a
failed watch creation. Being out of file descriptors typically
results in a cascading chain of errors which the process does
not usually survive.

This fix aims for minimal footprint. On failed watch creation
it does not render the cache fully unusable. If the process is
ever able to create new file descriptors again the cache also
becomes functional, but instead of autorefreshed mode it will
be in an 'always implicitly fully refreshed' mode.

Signed-off-by: Krisztian Litkey <[email protected]>

Author: klihub

pkg/cdi/cache.go

@@ -564,6 +564,14 @@ func (w *watch) update(dirErrors map[string]error, removed ...string) bool {
 		update bool
 	)
 
+	// If we failed to create an fsnotify.Watcher we have a nil watcher here
+	// (but with autoRefresh left on). One known case when this can happen is
+	// if we have too many open files. In that case we always return true and
+	// force a refresh.
+	if w.watcher == nil {
+		return true
+	}
+
 	for dir, ok = range w.tracked {
 		if ok {
 			continue