Merge branch 'master' of git.kawen.site:Lili/tlapbot

errors are warning, debug is for spam.

.gitignore

@@ -10,5 +10,3 @@ htmlcov/
 dist/
 build/
 *.egg-info/
-
-config.py

README.md

@@ -1,37 +1,88 @@
 # Tlapbot
-Tlapbot is an [Owncast](https://owncast.online/) bot, aiming to add the feature of channel points and
+Tlapbot is an [Owncast](https://owncast.online/) bot that adds channel points and
-channel point redeems to Owncast.
+channel point redeems to your Owncast page.
 
-This bot is currently in-development. The goal is to have a feature set on par
+This bot is currently in-development. The goal is to have an experience similar
-with Twitch channel points, while making use of Owncast webhooks and especially
+to [Twitch channel points](https://help.twitch.tv/s/article/viewer-channel-point-guide) by making use of [Owncast webhooks](https://owncast.online/thirdparty/webhooks/) and 
 [External actions](https://owncast.online/thirdparty/actions/).
 ## Features
-Currently, the bot gives points to everyone in chat -- the interval can be
+The bot gives points to everyone in chat -- 10 points every 10 minutes by
-configured in the config, as well as the amount of points given.
+default, but the time interval and amount of points can be changed.
 
-The users in chat can then use their points on redeems. The bot currently
+The users in chat can then use their points on redeems -- rewards like "choose my
-only has one hardcoded redeem, but I'd like to make this configurable,
+background music", "choose what level to play next", "react to this video" etc.
-so that every Owncast streamer can set up their own redeems that best fit
+You can configure redeems to fit your stream and the activities you're
-their stream.
+doing.
 
 The redeems then show on a "Redeems dashboard" that everyone can view
-at the flask server's URL, which can be included in Owncast
+as an External Action on the Owncast stream, or at its standalone URL.
-as an External action, a single button that displays information about
+This allows easy browsing of active challenges and recent redeems.
-recent redeems.
+### Tlapbot bot commands
+Tlapbot has these basic commands:
+- `!help` sends a help string in the chat, explaining how tlapbot works.
+- `!points` shows a chatter how many points they have.
+- `!name_update` is a special debug command, to be used with the user's name displays wrong in the redeem dashboard. Normally, it shouldn't have to be used at all, as display names get updated automatically when the bot is running.
+
+Tlapbot also automatically adds a command for each redeem in the redeems file.
+### Tlapbot redeems types
+Tlapbot currently supports three different redeem types.
+#### List
+List redeems are basic redeems, most similar to the ones on Twitch.
+
+Every time a chatter redeems a List redeem, the redeem gets added to the list of recent redeems with a timestamp, similar to how redeems on Twitch get added to the [Twitch redeem queue](https://help.twitch.tv/s/article/making-the-most-of-channel-points?language=en_US#manage).
+
+Unlike the Note redeems, chatters can't write messages to send along with their List redeems, so make sure the redeem makes sense on its own, like "stop talking for one minute", or "drop your weapon", etc.
+#### Note
+Note redeems are like List redeems, they get added to the list of recent redeems.
+
+Unlike the List redeems, chatters can add a message to their Note redeems, so this is the ideal type for open-ended redeems like "choose what character I play as next", "choose what song to play next", etc.
+#### Counter
+Counter is a unique redeem type, in that it doesn't show up in the list of recent redeems, and counters don't list the people who redeemed them. This redeem type is good for any rewards or incentives where the important thing isn't "who redeemed it?" but rather "how many people redeemed it?"
+
+Instead, the tlapbot dashboard keeps a number for each "counter", and each redeems adds +1 to it.
+
+Counter redeems can be used to gauge interest, tally up votes, or to keep track of how many emotes should be added to an OBS scene.
+
+### Tlapbot dashboard
+Tlapbot dashboard is a standalone page available at `/dashboard`, made to be easily viewable as an owncast external action. The Tlapbot dashboard shows all redeems and active counters.
+
+Counters are at the top, followed by a chronological list of recent List and Note redeems.
+
+Tlapbot dashboard also shows the chatter's points balance when they open it as an external action.
+
+![Tlapbot dashboard](https://ak.kawen.space/media/67c1ac6ed0d2f4efb09937c1cbfe864102182e990796853507860f50db7ff5f5.png)
+
+*Tlapbot dashboard when viewed as an external action.*
+#### Redeems help tab
+The dashboard also has a "Redeems help" tab. It shows an explanation of redeem types,
+and lists all active redeems, along with their price, type and description.
+## License & Contributions
+Tlapbot as it currently is does not come with a license. If you're a content creator, streamer, vtuber, etc. I'll be happy to give you permission to use Tlapbot, or make changes that'd fit your stream.
+
+I didn't make Tlapbot available under a permissive or a free software license, as Owncast is also used by some religious groups, extremist individuals, and dubious corporations to self-host their streams, and I do not want for them to use the bot.
 ## Setup
+Tlapbot requires Python 3, probably a fairly recent version of it too. (My live instance runs on Python 3.9.2.)
+
 The Python prerequisites for running tlapbot are the libraries `flask`,
-`requests` and `apscheduler`.
+`requests` and `apscheduler`. If you follow the installation steps below,
-### Dev setup (from git repository)
+they should automatically be installed if you don't have them.
+
+You can also run Tlapbot in a [Python virtual environment](https://flask.palletsprojects.com/en/2.2.x/installation/#virtual-environments).
+### Install from git repo (as a folder)
 1. Clone the repository.
 2. Run `pip install -e .` in the root folder. This will install tlapbot
-as a package in editable more, along with all its prerequisites.
+as a package in editable more.
-3. Initialize db:
+3. Set the `FLASK_APP` environment variable.
+    ```bash
+    export FLASK_APP=tlapbot
+    ```
+4. Initialize the database:
     ```bash
     python -m flask init-db
     ```
-4. Create an `instance/config.py` file and fill it in as needed.
+5. Create a `instance/config.py` file and fill it in as needed.
 Default values are included in `tlapbot/default_config`, and values in
-`config.py` overwrite them. (The database also lives in the `instance` folder
+`config.py` overwrite them. (The database also lives in the instance folder
 by default.)
 
     Tlapbot might not work if you don't overwrite these:
@@ -40,35 +91,59 @@ by default.)
     OWNCAST_ACCESS_TOKEN # get one from owncast instance
     OWNCAST_INSTANCE_URL # default points to localhost owncast on default port
     ```
-## Owncast setup
+6. OPTIONAL: Create an `instance/redeems.py` file and add your custom redeems.  
-In Owncast, navigate to the admin interface at `/admin`,
+  If you don't add a redeems file, the bot will initialize the default redeems from `tlapbot/default_redeems.py`.  
+  More details on how to write the config and redeems files are written later in the readme.
+
+This installation is fine both for just running Tlapbot as it is, but it also works as a dev setup if you want to make changes or contribute.
+
+Updating should be as easy as `git pull`ing the new version.
+## Owncast  configuration
+In the Owncast web interface, navigate to the admin interface at `/admin`,
 and then go to Integrations.
 ### Access Token
-In Access Tokens, generate an Access Token to put in
+In the Access Tokens tab, generate an Access Token to put in
-`instance/config.py`. The bot needs both the "send chat messages" and "perform administrative actions"
+the config file in the instance folder. The bot needs both the "send chat messages" and "perform administrative actions"
 permissions, since getting the list of all connected chat users is an administrator-only
 action.
 ### Webhook
-In webhooks, create a Webhook, and point it at your bot's URL with
+In the webhooks tab, create a Webhook, and point it at your bot's URL with
 `/owncastWebhook` added.
 
-In debug, this will be something like `localhost:5000/owncastWebhook`,
+In debug, this will be something like `localhost:5000/owncastWebhook`. If you're not running the debug Owncast instance and bot on the same machine,
-or, if you're not running the debug Owncast instance and bot on the same machine,
 you can use a tool like [ngrok](https://ngrok.com/)
-to redirect the Owncast traffic to your `localhost`.
+to redirect Owncast traffic to your `localhost`.
 ### External Action
 In External Actions, point the external action to your bot's URL with `/dashboard` added.
 
-In debug, pointing the External Action to an address like `localhost:5000/dashboard` might not work because your localhost address doesn't provide https, which owncast requires.
+**External Actions only work with https. Your server will need to support SSL and
+https connections for this part to work.**
 
+In development, a `localhost` address will not work with External Actions, since it doesn't provide https.
 If you use [ngrok](https://ngrok.com/) to redirect Owncast traffic to localhost,
 it will work because the ngrok connection is https.
 
-**Example:**
+**External Action config example:**
 ```
 URL: MyTlapbotServer.com/dashboard
 Action Title: Redeems Dashboard
 ```
+#### Note about https and reverse proxying
+Since External Actions require a secure https connection (for the tlapbot dashboard to work), you will need to set up a reverse proxy for tlapbot on your server. I'm not including much information about it here, since some knowledge of the topic is required to set up Owncast itself.
+
+The Owncast documentation about SSL and Reverse proxying is here: https://owncast.online/docs/sslproxies/
+
+If your followed the [Owncast recommendation to use Caddy](https://owncast.online/docs/sslproxies/caddy/) you'd only need to include this in your caddyfile to make the tlapbot dashboard work:
+
+```
+MyTlapbotServer.com {
+        reverse_proxy localhost:8000
+}
+```
+then MyTlapbotServer.com/owncastWebhook is the URL for webhooks,
+and MyTlapbotServer.com/dashboard is the URL for the dashboard.
+
+(And, obviously, you'd need to own the MyTlapbotServer.com domain, and have an A record pointing to your server's IP address.)
 ## Running the bot
 ### Running in debug:
 Set the FLASK_APP variable:
@@ -83,14 +158,105 @@ Run the app (in debug mode):
 ```bash
 python -m flask --debug run 
 ```
-### Running in prod:
+### Running in production:
-To be added when I actually run a prod version of the bot.
+Set the FLASK_APP variable:
-## Config
+```bash
-Values you can include in `instance/config.py` to change how the bot behaves.
+export FLASK_APP=tlapbot
-### Channel points interval and amount
+```
-`POINTS_CYCLE_TIME` decides how often channel points are given to users in chat,
+
+Using the flask debug server for running apps for non-development purposes is not recommended. Rather, you should be using a proper [Python WSGI server](https://flask.palletsprojects.com/en/2.2.x/deploying/).
+On my own live owncast instance, I use gunicorn.
+
+Install gunicorn (if you don't have it installed):
+```bash
+pip install gunicorn
+```
+Run the app (with gunicorn):
+```bash
+gunicorn -w 1 'tlapbot:create_app()'
+```
+
+You should also set `GUNICORN` in your `config.py` file to `True` to see Tlapbot logs in your gunicorn output. By default, gunicorn will only show `error` and `critical` warnings, but you can set the `--log-level` argument when running the app to set it to `warning`, `debug` or `info` too.
+
+```bash
+gunicorn -w 1 --log-level debug 'tlapbot:create_app()'
+```
+
+**⚠️WARNING:** Because of the way the scheduler is initialized in the project,
+I recommend running tlapbot with only one gunicorn worker. (`-w 1`)
+
+If you use multiple workers, each worker sets up its own scheduler, and then users
+get given points by each worker. (So running the app with `-w 4` means users get four times as many points than listed in the config.)
+
+I'd like to fix this shortcoming of tlapbot at some point in the future (so that it can take advantage of extra workers), but for now it's broken like this.
+
+## CLI commands: updating config + redeems
+Tlapbot comes with a few Click CLI commands. The commands let you clear out counters and the redeems dashboard.
+
+#### init-db
+The init-db command initializes the database.
+
+**This command should only be run when first installing tlapbot.**
+#### clear-queue
+The clear-queue command clears the redeem queue and resets all active counters to zero.
+You should run this command if you're about to start a new stream, and want to start with empty counters and queue.
+
+```bash
+python -m flask clear-queue
+```
+
+**If you've changed the config, and want new/different counters to work, you should run the `refresh-counters` command first.**
+#### refresh-counters
+This command deletes old counters that are no longer in the config file, and then adds all counters from the config file.
+You should run this command every time you've added or removed counters from `redeems.py`.
+```bash
+python -m flask refresh-counters
+```
+This command only changes counters, so if you want to clear the queue with list and note redeems too, you should run `clear-queue` after it.
+## Configuration files
+### config.py
+Values you can include in `config.py` to change how the bot behaves.
+
+(`config.py` should be in the instance folder: `/instance/config.py` for folder install.)
+#### Mandatory
+Including these values is mandatory if you want tlapbot to work.
+- `SECRET_KEY` is your secret key. Get one from running `python -c 'import secrets; print(secrets.token_hex())'`
+- `OWNCAST_ACCESS_TOKEN` is the owncast access token that owncast will use to get list of users in chat. Generate one in your owncast instance.
+- `OWNCAST_INSTANCE_URL` is the full URL of your owncast instance, like `"http://MyTlapbotServer.com"`
+#### Optional
+Including these values will overwrite their defaults from `/tlapbot/default_config.py`.
+- `POINTS_CYCLE_TIME` decides how often channel points are given to users in chat,
 in seconds. 
+- `POINTS_AMOUNT_GIVEN` decides how many channel points users receive.
+- `LIST_REDEEMS` if `True`, all redeems will be listed after the `!help` command in chat.
+This makes the !help output quite long, so it's `False` by default.
+- `GUNICORN` if `True`, sets logging to use gunicorn's logger. Only set this to True if you're using Gunicorn to run tlapbot.
+#### Example config:
+An example to show what your config like could look like
+```python
+SECRET_KEY= # string with secret key would be here.
+OWNCAST_ACCESS_TOKEN="5AT0gbe9ZuzDunsBG0rcwfalQNTi3fvV70NPvvQHk3I="
+OWNCAST_INSTANCE_URL="http://MyTlapbotServer.com"
+POINTS_CYCLE_TIME=300
+LIST_REDEEMS=True
+```
+### redeems.py
+`redeems.py` is a file where you define all your custom redeems. Tlapbot will work without it, but it will load a few default, generic redeems from `tlapbot/default_redeems.py`.
 
-`POINTS_AMOUNT_GIVEN` decides how many channel points users receive.
+(`redeems.py` should be in the instance folder: `/instance/redeems.py` for folder install.)
+#### Default `redeems.py`:
+```python
+REDEEMS={
+    "hydrate": {"price": 60, "type": "list"},
+    "lurk": {"price": 1, "type": "counter", "info": "Let us know you're going to lurk."},
+    "react": {"price": 200, "type": "note", "info": "Attach link to a video for me to react to."},
+    "request": {"price": 100, "type": "note", "info": "Request a level, gamemode, skin, etc."}
+}
+```
+#### File format
+`redeems.py` is a config file with just a `REDEEMS` key, that assigns a dictionary of redeems to it.
+Each dictionary entry is a redeem, and the dictionary keys are strings that decides the chat command for the redeem. The value is another dictionary that needs to have entries for `"price"`, `"type"` and optionally `"info"`.
 
-By default, everyone receives 10 points every 600 seconds (10 minutes).
+- `"price"` value should be an integer that decides how many points the redeem will cost.
+- `"type"` value should be either `"list"`, `"counter"` or `"note"`. This decided the redeem's type, and whether it will show up as a counter at the top of the dashboard or as an entry in the "recent redeems" chart.
+- `"info"` value should be a string that describes what the command does. It's optional, but I recommend writing one for all `"list"` and `"note"` redeems (so that chatters know that they should write a note).

setup.py

@@ -2,7 +2,7 @@ from setuptools import find_packages, setup
 
 setup(
     name='tlapbot',
-    version='0.6.1',
+    version='1.0.0',
     packages=find_packages(),
     include_package_data=True,
     install_requires=[

tlapbot/__init__.py

@@ -1,4 +1,5 @@
 import os
+import logging
 from flask import Flask
 from apscheduler.schedulers.background import BackgroundScheduler
 from tlapbot.db import get_db
@@ -24,6 +25,12 @@ def create_app(test_config=None):
     app.config.from_pyfile('config.py', silent=True)
     app.config.from_pyfile('redeems.py', silent=True)
 
+    # Make logging work for gunicorn-ran instances of tlapbot.
+    if app.config['GUNICORN'] is True:
+        gunicorn_logger = logging.getLogger('gunicorn.error')
+        app.logger.handlers = gunicorn_logger.handlers
+        app.logger.setLevel(gunicorn_logger.level)
+
     # prepare webhooks and redeem dashboard blueprints
     from . import owncast_webhooks
     from . import owncast_redeem_dashboard
@@ -40,7 +47,10 @@ def create_app(test_config=None):
     def proxy_job():
         with app.app_context():
             if is_stream_live():
+                app.logger.info("Stream is LIVE. Giving points to chat.")
                 give_points_to_chat(get_db())
+            else:
+                app.logger.info("Stream is NOT LIVE. (Not giving points to chat.)")
 
     # start scheduler that will give points to users
     points_giver = BackgroundScheduler()

tlapbot/default_config.py

@@ -4,3 +4,4 @@ OWNCAST_INSTANCE_URL='http://localhost:8080'
 POINTS_CYCLE_TIME=600
 POINTS_AMOUNT_GIVEN=10
 LIST_REDEEMS=False
+GUNICORN=False

tlapbot/owncast_helpers.py

@@ -1,15 +1,12 @@
 from flask import current_app
 import requests
 from sqlite3 import Error
-from flask.cli import with_appcontext
-from tlapbot.db import get_db
 
 
 # # # requests stuff # # #
 def is_stream_live():
     url = current_app.config['OWNCAST_INSTANCE_URL'] + '/api/status'
     r = requests.post(url)
-    print(r.json()["online"])
     return r.json()["online"]
 
 
@@ -41,8 +38,8 @@ def read_users_points(db, user_id):
         )
         return cursor.fetchone()[0]
     except Error as e:
-        print("Error occured reading points:", e.args[0])
+        current_app.logger.error("Error occured reading points:", e.args[0])
-        print("To user:", user_id)
+        current_app.logger.error("To user:", user_id)
 
 
 def read_all_users_with_username(db, username):
@@ -54,8 +51,9 @@ def read_all_users_with_username(db, username):
         users = cursor.fetchall()
         return users
     except Error as e:
-        print("Error occured reading points from username:", e.args[0])
+        current_app.logger.error("Error occured reading points from username:", e.args[0])
-        print("To user:", username)
+        current_app.logger.error("To user:", username)
+
 
 def give_points_to_user(db, user_id, points):
     try:
@@ -65,8 +63,8 @@ def give_points_to_user(db, user_id, points):
         )
         db.commit()
     except Error as e:
-        print("Error occured giving points:", e.args[0])
+        current_app.logger.error("Error occured giving points:", e.args[0])
-        print("To user:", user_id, "  amount of points:", points)
+        current_app.logger.error("To user:", user_id, "  amount of points:", points)
 
 
 def use_points(db, user_id, points):
@@ -78,8 +76,8 @@ def use_points(db, user_id, points):
         db.commit()
         return True
     except Error as e:
-        print("Error occured using points:", e.args[0])
+        current_app.logger.error("Error occured using points:", e.args[0])
-        print("From user:", user_id, "  amount of points:", points)
+        current_app.logger.error("From user:", user_id, "  amount of points:", points)
         return False
 
 
@@ -93,8 +91,8 @@ def user_exists(db, user_id):
             return False
         return True
     except Error as e:
-        print("Error occured checking if user exists:", e.args[0])
+        current_app.logger.error("Error occured checking if user exists:", e.args[0])
-        print("To user:", user_id)
+        current_app.logger.error("To user:", user_id)
 
 
 def add_user_to_database(db, user_id, display_name):
@@ -110,7 +108,7 @@ def add_user_to_database(db, user_id, display_name):
                 "INSERT INTO points(id, name, points) VALUES(?, ?, 10)",
                 (user_id, display_name)
             )
-        if user is not None and user[1] == None:
+        if user is not None and user[1] is None:
             cursor.execute(
                 """UPDATE points
                 SET name = ?
@@ -119,8 +117,8 @@ def add_user_to_database(db, user_id, display_name):
             )
         db.commit()
     except Error as e:
-        print("Error occured adding user to db:", e.args[0])
+        current_app.logger.error("Error occured adding user to db:", e.args[0])
-        print("To user:", user_id, display_name)
+        current_app.logger.error("To user:", user_id, display_name)
 
 
 def change_display_name(db, user_id, new_name):
@@ -131,8 +129,8 @@ def change_display_name(db, user_id, new_name):
         )
         db.commit()
     except Error as e:
-        print("Error occured changing display name:", e.args[0])
+        current_app.logger.error("Error occured changing display name:", e.args[0])
-        print("To user:", user_id, new_name)
+        current_app.logger.error("To user:", user_id, new_name)
 
 
 def add_to_counter(db, counter_name):
@@ -143,8 +141,8 @@ def add_to_counter(db, counter_name):
         )
         db.commit()
     except Error as e:
-        print("Error occured adding to counter:", e.args[0])
+        current_app.logger.error("Error occured adding to counter:", e.args[0])
-        print("To counter:", counter_name)
+        current_app.logger.error("To counter:", counter_name)
 
 
 def add_to_redeem_queue(db, user_id, redeem_name, note=None):
@@ -155,10 +153,8 @@ def add_to_redeem_queue(db, user_id, redeem_name, note=None):
             )
         db.commit()
     except Error as e:
-        print("Error occured adding to redeem queue:", e.args[0])
+        current_app.logger.error("Error occured adding to redeem queue:", e.args[0])
-        print("To user:", user_id, " with redeem:", redeem_name, "with note:", note)
+        current_app.logger.error("To user:", user_id, " with redeem:", redeem_name, "with note:", note)
-
-
 
 
 def all_counters(db):
@@ -168,7 +164,7 @@ def all_counters(db):
         )
         return cursor.fetchall()
     except Error as e:
-        print("Error occured selecting all counters:", e.args[0])
+        current_app.logger.error("Error occured selecting all counters:", e.args[0])
 
 
 def pretty_redeem_queue(db):
@@ -181,7 +177,7 @@ def pretty_redeem_queue(db):
         )
         return cursor.fetchall()
     except Error as e:
-        print("Error occured selecting pretty redeem queue:", e.args[0])
+        current_app.logger.error("Error occured selecting pretty redeem queue:", e.args[0])
 
 
 def whole_redeem_queue(db):
@@ -191,7 +187,7 @@ def whole_redeem_queue(db):
         )
         return cursor.fetchall()
     except Error as e:
-        print("Error occured selecting redeem queue:", e.args[0])
+        current_app.logger.error("Error occured selecting redeem queue:", e.args[0])
 
 
 def remove_duplicate_usernames(db, user_id, username):
@@ -200,8 +196,8 @@ def remove_duplicate_usernames(db, user_id, username):
             """UPDATE points
             SET name = NULL
             WHERE name = ? AND NOT id = ?""",
-        (username, user_id)
+            (username, user_id)
         )
         db.commit()
     except Error as e:
-        print("Error occured removing duplicate usernames:", e.args[0])
+        current_app.logger.error("Error occured removing duplicate usernames:", e.args[0])

tlapbot/owncast_webhooks.py

@@ -1,5 +1,4 @@
 from flask import Flask, request, json, Blueprint, current_app
-from sqlite3 import Error
 from tlapbot.db import get_db
 from tlapbot.owncast_helpers import (add_user_to_database, change_display_name,
         user_exists, send_chat, read_users_points, remove_duplicate_usernames)
@@ -30,8 +29,8 @@ def owncast_webhook():
     elif data["type"] == "CHAT":
         user_id = data["eventData"]["user"]["id"]
         display_name = data["eventData"]["user"]["displayName"]
-        print(f'New chat message from {display_name}:')
+        current_app.logger.debug(f'New chat message from {display_name}:')
-        print(f'{data["eventData"]["body"]}')
+        current_app.logger.debug(f'{data["eventData"]["body"]}')
         if "!help" in data["eventData"]["body"]:
             send_help()
         elif "!points" in data["eventData"]["body"]: