"getport" sometimes throwing an error

[tarasromil]

Versions

• NodeJS: 16.20.2
• mongodb-memory-server-*: 9.0.1
• mongodb(the binary version): 6.0.10
• mongodb(the js package): 5.9.0
• system: MacOS 14.1

package: mongo-memory-server

What is the Problem?

When I'm trun to run test using "vitest@0.34.6" in concurrency mode I have an error that port already in use. No always, but pretty often

Starting the MongoMemoryServer Instance failed, enable debug log for more information. Error:
 StdoutInstanceError: Port "15490" already in use

Want to notice that there is no error on v8.16.0

Code Example

globalThis.__MONGO_DB__ = await MongoMemoryServer.create();

Debug Output

stderr | unknown test
Starting the MongoMemoryServer Instance failed, enable debug log for more information. Error:
 StdoutInstanceError: Port "52929" already in use
    at MongoInstance.checkErrorInLine (/.../node_modules/.pnpm/mongodb-memory-server-core@9.0.1/node_modules/mongodb-memory-server-core/src/util/MongoInstance.ts:641:9)
    at MongoInstance.stdoutHandler (/.../node_modules/.pnpm/mongodb-memory-server-core@9.0.1/node_modules/mongodb-memory-server-core/src/util/MongoInstance.ts:615:10)
    at Socket.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at Pipe.onStreamRead (node:internal/stream_base_commons:190:23)

Do you know why it happenes?

Looks like there is a race condition after replacing "get-port" to own implementation

[hasezoey]

i dont know how vitest's concurrency works or how / where you create your server.
i also dont know why the new implementation would fail when the old one would not in the same scenario.

port acquisition and keeping is a fickle process, because you cant just "give me a free port and keep it on-hold until used" and also cannot be marked as "used for later", like it is tested that it is free with a small http server, which is closed to free the port so that it can be used by mongod, but in that small time another process may just get that port

[AndreasGalster2]

I just got here because I have the same issue. Since there's currently no solution, I'm thinking maybe https://www.npmjs.com/package/promise-retry will do the trick to create a server in case the port fails. I haven't tried this yet but that might work at least as a temporary fix.

/edit:

    import promiseRetry from 'promise-retry';

    const promiseRetryOptions = {
      retries: 3,
      factor: 2,
      minTimeout: 50,
      maxTimeout: 10000,
    };

    return promiseRetry((retry, number) => {
      console.log(`Creating MongoClient - Retrying if port is unavailable - Try Number: ${number}`);

      return mongoose.connect(url, { useNewUrlParser: true });
    }, promiseRetryOptions);

I've done a few runs now and it seems I no longer have port issues. No guarantees here but out of 5 runs, they all passed... Before every 2nd run or so failed.

[JacksonUtsch]

I seem to getting the same port issue sometimes. Oddly enough also on macOS 14.1. but mongodb-memory-server v8.13.0. Will try newer version.

[trevorah]

We've had the same issue with v9.0.1 on macos and on GHA linux build machines, where v8.13.0 was fine.

We ended up with this workaround:

import getPort from 'get-port';
import retry from 'async-retry';

const port = await getPort();
const mongo = await retry(
  () =>
    MongoMemoryReplSet.create({
      ...this.testEnvironmentOptions,
      instanceOpts: [{ port }],
      replSet: { count: 2 },
    }),
  {
    retries: 3,
  },
);

[JacksonUtsch]

Well, it seems I have interesting results. Updating to v9.0.1 seems to made this issue much less frequent.

[hasezoey]

@JacksonUtsch @trevorah, what exact errors are you seeing, are you seeing a Port is already in use(by the mongod log) error or a Max port tries exceeded error?

i also dont know what is so much different from the get-port implementation, does someone maybe have some suggestions? code here

[JacksonUtsch]

Port "50113" already in use
  at MongoInstance.checkErrorInLine (node_modules/mongodb-memory-server-core/src/util/MongoInstance.ts:611:9)
  at MongoInstance.stdoutHandler (node_modules/mongodb-memory-server-core/src/util/MongoInstance.ts:585:10)

In the code referenced I think we need a lock of sorts for PORTS_CACHE. If we could await read and writing of that value. Does that sound about right?

[hasezoey]

In the code referenced I think we need a lock of sorts for PORTS_CACHE. If we could await read and writing of that value. Does that sound about right?

do you mean some kind of mutex or do you mean a cache file with lock?

also what is your use-case, replset or just many severs?

[JacksonUtsch]

A mutex yes. Running many e2e tests from separate files.

[hasezoey]

Running many e2e tests from separate files.

what is your testing environment?

(example jest) the problem with many concurrent tests and each having their own server is that those modules are isolated and cannot speak to each-other without some kind of cache file (so a normal mutex would not work), but i didnt want to increase complexity compared to the old get-port (that is why it is recommended to start a server in globalSetup)

[JacksonUtsch]

Yes, using jest for testing here.

Hmm, I thought there was only a single instance of PORTS_CACHE defined at the file level and they would all use the same? Ex: A singleton or global variable with locking.

[hasezoey]

Hmm, I thought there was only a single instance of PORTS_CACHE defined at the file level and they would all use the same? Ex: A singleton or global variable with locking.

yes it is a singleton, but the problem is that they are isolated instances, each jest worker can be seen as its own nodejs process, so a singleton is not shared across those processes (at least to my knowledge).
basically meaning that each test file is executed as a "new nodejs process"

[JacksonUtsch]

Ahh, good to know. Then a temp file might be the way to go.