My Bluetooth speaker works perfectly after pairing it, but when I disconnect the speaker and then want to reconnect it later, no sound is coming from the paired Bluetooth audio device anymore unless I reboot.

Never had any problems with my bluetooth mouse.
The Bluetooth speaker:
is listed in Control Panel
is connected
is set up as default
has been reinstalled
updates up to date
Device troubleshooter couldn't identify the problem
Control panel > properties device > services: audio sink & remote control

Method 1:
Check if the Bluetooth service is running.
a) Press Windows Key + R, type services.msc and press Enter.
b) Search for Bluetooth support service.
c) Check if it is started, if not Right click on the service and click on Start.

Method 2:
Reinstall the Bluetooth driver following these steps:
a) Press Windows key + X and select “Device Manager”
b) Expand “Bluetooth” and right-click on the driver.
c) Select “Uninstall”, check marked “delete the software for the device” and select “OK” (after that, you can see there is no “Bluetooth” expansion on "Device Manager")
d) Select “Update Driver Software”.
e) Restart the computer.

Method 3:
Restart Bluetooth wireless connectivity.
Steps:
a. From the Start screen, swipe in from the right edge of the screen, and then tap or click Settings.
b. Tap or click Change PC settings.
c. Tap or click PC and devices, and then tap or click Bluetooth.
d. Under Bluetooth, tap or click to turn Bluetooth Off, and then tap or click again to turn Bluetooth On.