RemoteConfig NullPointerException crash

[fanwgwg]

[READ] Step 1: Are you in the right place?

Issues filed here should be about bugs in the code in this repository.
If you have a general question, need help debugging, or fall into some
other category use one of these other channels:

• For general technical questions, post a question on StackOverflow
  with the firebase tag.
• For general Firebase discussion, use the firebase-talk
  google group.
• For help troubleshooting your application that does not fall under one
  of the above categories, reach out to the personalized
  Firebase support channel.

[REQUIRED] Step 2: Describe your environment

• Android Studio version: Android Studio Iguana | 2023.2.1
• Firebase Component: config
• Component version: com.google.firebase:firebase-bom:32.7.3

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:

After upgrading to firebase bom 32.7.3, I'm seeing this remote-config crash on Crashlytics. Mostly when the app is in background, as shown in the screenshot below.

Fatal Exception: java.lang.NullPointerException: Attempt to invoke interface method 'void com.android.okhttp.internal.http.HttpStream.writeRequestHeaders(com.android.okhttp.Request)' on a null object reference
       at com.android.okhttp.internal.http.HttpEngine.readResponse(HttpEngine.java:606)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:475)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:411)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:248)
       at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:211)
       at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:30)
       at com.google.firebase.remoteconfig.internal.ConfigRealtimeHttpClient.lambda$closeRealtimeHttpStream$2(ConfigRealtimeHttpClient.java:605)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
       at com.google.firebase.concurrent.CustomThreadFactory.lambda$newThread$0(CustomThreadFactory.java:47)
       at java.lang.Thread.run(Thread.java:1012)

[pacoarjona]

In my app I also have the same errors, with that version of BoM. Luckily I have a control in the app to prevent that if there are errors in threads other than the main thread, the app does not crash... it is literally saving my life in this case

[danasilver]

Hi there, thanks for the reports. I'm sorry you're running into this. I believe the error is related to a change in BoM version 32.7.3 to proactively close the real-time Remote Config server connection when the app is in the background. (https://firebase.google.com/support/release-notes/android#remote-config_v21-6-2)

I haven't been able to reproduce the issue so far, though. If you could, it'd be helpful to see more information about how you're using Remote Config, particularly where in the app you're calling addOnConfigUpdateListener.

Thanks!

[pacoarjona]

Hello, this is my code. I have not been able to reproduce the error in my tests either. If you need anything else, don't hesitate to ask!

public static void init(Activity activity) {
        try {
            if (hasInit || isLoading || activity.isFinishing()) {
                return;
            }

            isLoading = true;

            setDefaultValues();

            int duration = UtilsSharedPreferences.getMinimumFetchIntervalInSeconds();
            FirebaseRemoteConfigSettings configSettings = new FirebaseRemoteConfigSettings.Builder()
                    .setMinimumFetchIntervalInSeconds(duration).build();

            mFirebaseRemoteConfig = FirebaseRemoteConfig.getInstance();
            mFirebaseRemoteConfig.setConfigSettingsAsync(configSettings);
            mFirebaseRemoteConfig.setDefaultsAsync(defaultValues);

            mFirebaseRemoteConfig.fetchAndActivate().addOnCompleteListener(activity, task -> {
                isLoading = false;

                if (task.isSuccessful()) {
                    if (UtilsWiFi.isWIFIConnected() || UtilsWiFi.is4GConnected()) {
                        hasInit = true;
                        createRealTime();
                    }
                }
            });
        } catch (Throwable e) {
            exception(e);
        }
    }

    private static void createRealTime() {
        try {
            mFirebaseRemoteConfig.addOnConfigUpdateListener(new ConfigUpdateListener() {
                @Override
                public void onUpdate(@NonNull ConfigUpdate configUpdate) {
                    try {
                        mFirebaseRemoteConfig.activate();
                    } catch (Throwable e) {
                        exception(e); //Log exception (non-fatal) with crashlytics
                    }
                }

                @Override
                public void onError(@NonNull FirebaseRemoteConfigException error) {

                }
            });
        } catch (Throwable e) {
            exception(e); //Log exception (non-fatal) with crashlytics
        }
    }

The function that is causing the error is the following. Maybe catching any type of exception in the 'catch' is enough to avoid errors.

public void closeRealtimeHttpStream() {
    if (httpURLConnection != null) {
      // Network operations must be off the main thread.
      this.scheduledExecutorService.execute(
          () -> {
            httpURLConnection.disconnect();

            // Explicitly close the input stream due to a bug in the Android okhttp implementation.
            // See github.com/firebase/firebase-android-sdk/pull/808.
            try {
              httpURLConnection.getInputStream().close();
              if (httpURLConnection.getErrorStream() != null) {
                httpURLConnection.getErrorStream().close();
              }
            } catch (IOException | IllegalStateException e) { <---- //Replace by catch (Exception e)
              // HttpUrlConnection enforces a strict lifecycle. If the connection is closed before
              // the response is read, it'll throw an IllegalStateException. See docs:
              // https://android.googlesource.com/platform/external/okhttp/+/602d5e4/okhttp/src/main/java/com/squareup/okhttp/internal/http/HttpConnection.java#43
            }
          });

      setIsHttpConnectionRunning(false);
    }
  }

[danasilver]

Thanks for all that info @pacoarjona! That's really helpful.

Is init called from the MainActivity's onCreate or somewhere else in the app? Also, do you have a sense of percent of the app's users affected by this?

[SimonMarquis]

👋 Same issue here.
Here is our addOnConfigUpdateListener() call:

when (isOnline(context)) {
    true -> firebase.fetchAndActivate()
    false -> firebase.activate()
}

firebase.addOnConfigUpdateListener(
    object : ConfigUpdateListener {
        override fun onUpdate(configUpdate: ConfigUpdate) {
           configUpdate.log()
           firebase.activate()
        }
        override fun onError(error: FirebaseRemoteConfigException) = error.log()
    },
)

It is called in a androidx.startup.Initializer, after regular Firebase RemoteConfig initialization and before the Application.onCreate().

[fanwgwg]

Here's our setup:

    FirebaseRemoteConfig remoteConfig = getRemoteConfig();
    Futures.addCallback(
        FuturesUtils.taskToListenableFuture(remoteConfig.fetchAndActivate()),
        new FutureCallback<>() {
          @Override
          public void onSuccess(Boolean result) {
            if (Boolean.TRUE.equals(result)) {
              remoteConfig.addOnConfigUpdateListener(
                  new ConfigUpdateListener() {
                    @Override
                    public void onUpdate(@NonNull ConfigUpdate configUpdate) {
                      remoteConfig.activate();
                    }

                    @Override
                    public void onError(@NonNull FirebaseRemoteConfigException error) {
                            // no-op
                    }
                  });
            }
          }

          @Override
          public void onFailure(@NonNull Throwable t) {
                 // no-op
          }
        },
        MoreExecutors.directExecutor());

init was called in Application#onCreate. For us this is happening to about ~0.5% users, 86% in the background

[pacoarjona]

Thanks for all that info @pacoarjona! That's really helpful.

Is init called from the MainActivity's onCreate or somewhere else in the app? Also, do you have a sense of percent of the app's users affected by this?

Hello again! The code is called from the Splash activity of the app, that is, it is called at the start of the app and I have controlled that this code can only be executed once per session.

In my case, the error is affecting approximately 7% of the users. 71% (according to crashlytics) of the total were in the background at the time of the error. 2000 errors in approximately 1000 users in just 5 days since I uploaded the update to the Play Store.

Here you can see more types of errors in addition to the NPE. Please, in your correction make sure that you capture any type of exception, this is crazy... for now I am going to rollback and put the previous version of BoM while you solve the problem, I cannot continue with so many errors daily.

Non-fatal Exception: java.lang.IndexOutOfBoundsException
Invalid index 31, size is 31
java.util.ArrayList.throwIndexOutOfBoundsException (ArrayList.java:255)
java.util.ArrayList.get (ArrayList.java:308)
com.android.okhttp.internal.http.RawHeaders.toBytes (RawHeaders.java:294)
com.android.okhttp.internal.http.HttpTransport.writeRequestHeaders (HttpTransport.java:130)
com.android.okhttp.internal.http.HttpEngine.readResponse (HttpEngine.java:632)
com.android.okhttp.internal.http.HttpURLConnectionImpl.execute (HttpURLConnectionImpl.java:347)
com.android.okhttp.internal.http.HttpURLConnectionImpl.getResponse (HttpURLConnectionImpl.java:296)
com.android.okhttp.internal.http.HttpURLConnectionImpl.getInputStream (HttpURLConnectionImpl.java:179)
com.android.okhttp.internal.http.HttpsURLConnectionImpl.getInputStream (HttpsURLConnectionImpl.java:246)
com.google.firebase.remoteconfig.internal.ConfigRealtimeHttpClient.lambda$closeRealtimeHttpStream$2 (ConfigRealtimeHttpClient.java:605)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1112)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:587)
com.google.firebase.concurrent.CustomThreadFactory.lambda$newThread$0 (CustomThreadFactory.java:47)
java.lang.Thread.run (Thread.java:841)

Non-fatal Exception: java.util.NoSuchElementException
com.android.okhttp.internal.http.RouteSelector.next (RouteSelector.java:104)
com.android.okhttp.internal.http.HttpEngine.createNextConnection (HttpEngine.java:501)
com.android.okhttp.internal.http.HttpEngine.nextConnection (HttpEngine.java:484)
com.android.okhttp.internal.http.HttpEngine.connect (HttpEngine.java:466)
com.android.okhttp.internal.http.HttpEngine.sendRequest (HttpEngine.java:372)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute (HttpURLConnectionImpl.java:476)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse (HttpURLConnectionImpl.java:418)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream (HttpURLConnectionImpl.java:235)
com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream (DelegatingHttpsURLConnection.java:210)
com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream (HttpsURLConnectionImpl.java:25)
com.google.firebase.remoteconfig.internal.ConfigRealtimeHttpClient.lambda$closeRealtimeHttpStream$2 (ConfigRealtimeHttpClient.java:605)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1113)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:588)
com.google.firebase.concurrent.CustomThreadFactory.lambda$newThread$0 (CustomThreadFactory.java:47)
java.lang.Thread.run (Thread.java:818)

Non-fatal Exception: java.lang.AssertionError
java.io.EOFException
com.android.okhttp.okio.Buffer.clear (Buffer.java:764)
com.android.okhttp.okio.RealBufferedSource.close (RealBufferedSource.java:397)
com.android.okhttp.okio.InflaterSource.close (InflaterSource.java:126)
com.android.okhttp.okio.GzipSource.close (GzipSource.java:182)
com.android.okhttp.okio.RealBufferedSource.close (RealBufferedSource.java:396)
com.android.okhttp.okio.RealBufferedSource$1.close (RealBufferedSource.java:384)
com.google.firebase.remoteconfig.internal.ConfigRealtimeHttpClient.lambda$closeRealtimeHttpStream$2 (ConfigRealtimeHttpClient.java:605)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1133)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:607)
com.google.firebase.concurrent.CustomThreadFactory.lambda$newThread$0 (CustomThreadFactory.java:47)
java.lang.Thread.run (Thread.java:761)

Non-fatal Exception: java.lang.ArrayIndexOutOfBoundsException
length=65536; regionStart=118; regionLength=-84
com.google.android.gms.org.conscrypt.ArrayUtils.checkOffsetAndCount (:com.google.android.gms@240713022@24.07.13 (100400-607434947):47)
com.google.android.gms.org.conscrypt.ConscryptFileDescriptorSocket$SSLOutputStream.write (:com.google.android.gms@240713022@24.07.13 (100400-607434947):4)
com.android.okhttp.okio.Okio$1.write (Okio.java:81)
com.android.okhttp.okio.AsyncTimeout$1.write (AsyncTimeout.java:155)
com.android.okhttp.okio.RealBufferedSink.emitCompleteSegments (RealBufferedSink.java:176)
com.android.okhttp.okio.RealBufferedSink.writeUtf8 (RealBufferedSink.java:58)
com.android.okhttp.internal.http.Http1xStream.writeRequest (Http1xStream.java:371)
com.android.okhttp.internal.http.Http1xStream.writeRequestHeaders (Http1xStream.java:140)
com.android.okhttp.internal.http.HttpEngine.readResponse (HttpEngine.java:743)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute (HttpURLConnectionImpl.java:480)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse (HttpURLConnectionImpl.java:416)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.getErrorStream (HttpURLConnectionImpl.java:150)
com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getErrorStream (DelegatingHttpsURLConnection.java:97)
com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getErrorStream (HttpsURLConnectionImpl.java:26)
com.google.firebase.remoteconfig.internal.ConfigRealtimeHttpClient.lambda$closeRealtimeHttpStream$2 (ConfigRealtimeHttpClient.java:606)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1167)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:641)
com.google.firebase.concurrent.CustomThreadFactory.lambda$newThread$0 (CustomThreadFactory.java:47)
java.lang.Thread.run (Thread.java:764)

[danasilver]

Thanks again for the reports everyone - we're working on a patch to fix this. We'll update this issue once that's available

[danasilver]

Thanks for your patience! BoM version 32.7.4 (Remote Config version 21.6.3) has been released with a fix for this issue: https://firebase.google.com/support/release-notes/android#2024-03-07