Compare commits

...

18 commits
v1.5 ... master

Author SHA1 Message Date
Dryusdan fa1550e26a Merge branch 'improve-randomness' of framasky/masto-image-bot into master 2019-09-12 12:05:48 +02:00
Luc Didry e5e4938479
Improve local images randomness by using random.SystemRandom()
My bot @gracyimp@botsin.space gave me the same image twice this morning,
within a few seconds, so I asked myself how to improve the randomness.
Indeed, this allowed my bot to publish some images that never were
published!
2019-09-04 09:24:26 +02:00
Dryusdan 67849b7f9f Remove quote for unsplash_client_id 2019-06-22 10:49:11 +02:00
Tristan Le Chanony 38c1afa81a Improve README 2019-01-07 13:52:34 +01:00
Tristan Le Chanony d7287622b9 Improve unsplash mode 2019-01-07 13:50:35 +01:00
Dryusdan 1a8f9a381a Merge branch 'add-unsplash-random-source' of framasky/masto-image-bot into master 2019-01-06 15:09:13 +01:00
Dryusdan 995370b7ea Merge branch 'update-systemd-service' of framasky/masto-image-bot into master 2019-01-06 15:07:31 +01:00
Luc Didry cc3cbd2fe7
Add unsplash-random source
Fetches a random image from unsplash and toot it with author's attribution
2019-01-06 14:55:19 +01:00
Luc Didry 8aef6ae903
Update systemd documentation 2019-01-05 23:30:59 +01:00
Dryusdan 0063745171 Merge branch 'linting' of framasky/masto-image-bot into master 2019-01-05 15:40:41 +01:00
Dryusdan 8332357d9a Merge branch 'improve-readme' of framasky/masto-image-bot into master 2019-01-05 15:39:46 +01:00
Luc Didry 43acb49171
Improve README (translation, install deps…)
This commit is dedicated to Le Libre au quotidion, who is supporting me with Ğ1.
Many thanks ☺️
2019-01-05 14:26:16 +01:00
Luc Didry 97b2c6b01e
Lintings changes
1. replace tabs by spaces
2. align assignations
3. no trailing spaces/tabs
4. put a re.compile outside a function, to make it compiled one time
   only
2018-11-20 22:16:08 +01:00
Dryusdan 320c7056d0 Mettre à jour 'bot.py' 2018-08-18 01:16:37 +02:00
Dryusdan d9ef0dc3af Resize image (because mastodon not accept big picture) 2018-08-18 00:40:21 +02:00
Dryusdan cf905aac3c add renamer 2018-07-31 23:09:47 +02:00
Dryusdan fde735c88a correct respond in thread #10 2018-06-23 21:11:04 +02:00
Dryusdan 975001bd4f Remove uselessfile 2018-06-23 09:04:19 +02:00
5 changed files with 262 additions and 199 deletions

View file

@ -1,11 +1,18 @@
# masto-image-bot
Un bot qui récupère une image random en local et la publie
A bot that fetches a random local image and publish it on Mastodon.
Copiez le fichier `config.sample.txt` en `config.txt`, ajoutez le chemin de votre dossier image.
Remplissez le fichier `secrets/secrets.txt` et remplissez le avec les code que vous trouverez dans l'onglet développeur de votre compte Mastodon.
Copy the file `config.sample.txt` to `config.txt` and add the path to your images directory.
File the file `secrets/secrets.txt` with the codes you will find in the developper tab of your Mastodon account.
You can also register your bot on a Mastodon instance and get the needed codes with the help of the script [register-app](https://framagit.org/fiat-tux/hat-softwares/mastodon/register-app).
## Configure it
## Install the dependencies
```
pip3 install -r requirements.txt
```
## Configure the bot
Copy `config.sample.txt` to `config.txt` and replace data by your data.
@ -25,6 +32,7 @@ If you don't want any "spoiler text", just leave the line empty.
| limit | Limit send per minute per person | int |
| limit_hour | Limit send par hour per person | int |
| collection_url | URL of website you deserve image. `<collection>` is a variable who depend on collection.json (you can remove this variable) | string |
| unsplash_client_id | Access key of your Unsplash App (you can create it on api.unsplash.com ) | string |
Copy `blacklist.sample.json` to `blacklist.json` and replace or add accounts that should not receive any image
@ -41,7 +49,7 @@ optional arguments:
-h, --help show this help message and exit
-i, --img post image
-s SOURCE, --source SOURCE
Source of image [ local | distant ]
Source of image [ local | distant | unsplash-random ]
--stream stream user profile
```
@ -66,4 +74,14 @@ User=masto-bot
TimeoutSec=15
WorkingDirectory=/home/masto-bot/
ExecStart=/usr/bin/python3 bot.py --stream --source=local
[Install]
WantedBy=multi-user.target
```
Then do
```
systemctl daemon-reload
systemctl enable bot.service
systemctl start bot.service
```

View file

@ -1,20 +0,0 @@
from html.parser import HTMLParser
class TootHTMLParser(HTMLParser):
def __init__(self):
super().__init__()
self.txt = ""
def handle_data(self, data):
self.txt += str(data).lstrip().rstrip().lower() + " "
#
#
# content = ""
# with open("input") as f:
# content = f.readlines()
# content = set(content)
# parser = TootHTMLParser()
# for word in content:
# parser.feed(word)
# with open("output", "w+") as f:
# f.write(parser.txt)

384
bot.py
View file

@ -12,52 +12,53 @@ from utils.config import get_parameter, init_log, init_mastodon
from PIL import Image
from io import BytesIO
import requests, os, random, sys, time, json, logging, argparse, re
import requests, os, random, sys, time, json, logging, argparse, re, shutil
config_file = "config.txt"
secrets_filepath = get_parameter("secrets_filepath", config_file)
log_filepath = get_parameter("log_filepath", config_file)
blacklist_filepath = get_parameter("blacklist_filepath", config_file)
collection_filepath = get_parameter("collection_filepath", config_file)
log = init_log(log_filepath)
mastodon = init_mastodon(config_file, secrets_filepath)
log = init_log(log_filepath)
mastodon = init_mastodon(config_file, secrets_filepath)
blacklist_file = open(blacklist_filepath,'r')
BLACKLIST = json.loads(blacklist_file.read())
BLACKLIST = json.loads(blacklist_file.read())
blacklist_file.close()
mime_dict = {'.jpg': 'image/jpeg', '.jpe': 'image/jpeg', '.jpeg': 'image/jpeg', '.png': 'image/png', '.gif': 'image/gif'}
def post_img_local(mastodon, text, log, config):
img_path = get_parameter("img_path", config)
continu = True;
while continu:
file = random.choice(os.listdir(img_path+"/"))
secure_random = random.SystemRandom()
file = secure_random.choice(os.listdir(img_path+"/"))
if os.path.isdir(img_path+file):
img_path = img_path+file+"/"
else:
if ".zip" not in file:
if ".zip" not in file:
continu = False
#file, ext = os.path.splittext(img_path+"/"+file)
#if ext in mime_dict:
im = Image.open(img_path+"/"+file)
im = Image.open(img_path+ file)
width, height = im.size
if width > 1980:
ratio_img = width / 1980;
new_width = width / ratio_img
new_height = height / ratio_img
NEW_WIDTH = 2048
if width > 2048:
difference_percent = NEW_WIDTH / width
new_height = height * difference_percent
size = new_height, NEW_WIDTH
im = im.resize((int(NEW_WIDTH), int(new_height)))
im.save('resize_img.jpg')
file = "resize_img.jpg"
shutil.copyfile(file, "/tmp/"+file)
else:
log.debug("no resize")
shutil.copyfile(img_path+file, "/tmp/"+file)
image = im.resize((int(new_width), int(new_height)))
im.save(img_path+"/"+file)
image_byte = open("/tmp/"+file, "rb").read()
file, ext = os.path.splitext(file)
os.remove("/tmp/"+file+ext)
im.close()
image_byte = open(img_path+"/"+file, "rb").read()
file, ext = os.path.splitext(img_path+"/"+file)
#mime = mime_dict[str.lower(ext)]
try:
mime = mime_dict[str.lower(ext)]
except KeyError:
@ -68,180 +69,215 @@ def post_img_local(mastodon, text, log, config):
media_dict = mastodon.media_post(image_byte, mime)
return media_dict;
def post_unsplash_random_image(mastodon, log, config):
collection_url = get_parameter("collection_url", config)
unsplash_client_id = get_parameter("unsplash_client_id", config)
collecion_file = open(collection_filepath,'r')
collections = json.loads(collecion_file.read())
collecion_file.close()
count_collection = len(collections)-1
if count_collection > -1:
id_collection = randint(0,count_collection)
collection_url="&collections="+str(collections[id_collection])
else:
collection_url=''
response = requests.get("https://api.unsplash.com/photos/random?client_id="+unsplash_client_id+collection_url)
randim_json = json.loads(response.text)
randim_url = "{}&q=85&crop=entropy&cs=tinysrgb&w=2048&fit=max".format(randim_json['urls']['raw'])
img_response = requests.get(randim_url)
pattern = Image.open(BytesIO(img_response.content), "r").convert('RGB')
pattern.save('output.jpg')
media_dict = mastodon.media_post("output.jpg")
toot = "Shot by {} ({})\n{}".format(randim_json['user']['name'], randim_json['user']['links']['html'], randim_json['links']['html'])
return { 'media_dict': media_dict, 'toot': toot };
def post_img_distant(mastodon, text, log, config):
collection_url = get_parameter("collection_url", config)
collecion_file = open(collection_filepath,'r')
collections = json.loads(collecion_file.read())
collections = json.loads(collecion_file.read())
collecion_file.close()
count_collection = len(collections)-1
id_collection = randint(0,count_collection)
id_collection = randint(0,count_collection)
collection_url = collection_url.replace("<collection>", str(collections[id_collection]))
response = requests.get(collection_url)
pattern = Image.open(BytesIO(response.content), "r").convert('RGB')
response = requests.get(collection_url)
pattern = Image.open(BytesIO(response.content), "r").convert('RGB')
pattern.save('output.jpg')
media_dict = mastodon.media_post("output.jpg")
return media_dict;
cleanr = re.compile('<.*?>')
def cleanhtml(raw_html):
cleanr = re.compile('<.*?>')
cleantext = re.sub(cleanr, '', raw_html)
return cleantext
class BotListener(StreamListener):
def __init__(self, args):
self.args = args
# use only notification
def on_notification(self, notification):
def __init__(self, args):
self.args = args
# catch only mention in notification
if notification['type'] == 'mention':
log.debug("Got a mention")
if notification["account"]["bot"] == False:
sender = notification['account']['acct'] # Get sender name
if sender in BLACKLIST:
log.info("Service refused to %s" % sender)
return
sender_hour_filename = "limiter/hour/" + sender; # Forge file for limiter
sender_minute_filename = "limiter/minute/" + sender; # Forge file for limiter
if os.path.isfile(sender_hour_filename): # Check if file exist
log.debug("Sender file exist")
statbuf = os.stat(sender_hour_filename)
last_edit = int(statbuf.st_mtime)
ts = int(time.time())
if ts - last_edit > 3599: # check if file is modified 1 hour after last edition
log.debug("file is too old")
f = open(sender_hour_filename,'w')
f.write(str(1)) # reset counter
f.close()
can_continue = True
else:
log.debug("file is young")
f = open(sender_hour_filename,'r+')
limit = int(get_parameter("limit_hour", config_file))
number_of_mention = int(f.read())
if number_of_mention < limit: # limit of mention per hour is limit_hour
log.debug("Sender have less of limit requests")
f.seek(0)
f.write(str(number_of_mention + 1))
can_continue = True
else:
log.debug("Sender have more of limit requests")
can_continue = False # if number of mention is for, user can't receive anything
f.close()
else: # File not exist, create it and initialise it
log.debug("Sender file not exist")
f = open(sender_hour_filename,"w+")
f.write(str(1))
f.close()
can_continue = True
if can_continue:
if os.path.isfile(sender_minute_filename): # Check if file exist
log.debug("Sender file exist")
statbuf = os.stat(sender_minute_filename)
last_edit = int(statbuf.st_mtime)
ts = int(time.time())
if ts - last_edit > 59: # check if file is modified 1 minute after last edition
log.debug("file is too old")
f = open(sender_minute_filename,'w')
f.write(str(1)) # reset counter
f.close()
can_continue = True
else:
log.debug("file is young")
f = open(sender_minute_filename,'r+')
limit = int(get_parameter("limit", config_file))
number_of_mention = int(f.read())
if number_of_mention < limit: # limit of mention per minute is 4
log.debug("Sender have less of limit requests")
f.seek(0)
f.write(str(number_of_mention + 1))
can_continue = True
else:
log.debug("Sender have more of limit requests")
can_continue = False # if number of mention is for, user can't receive anything
file = open(sender_hour_filename,'r+')
number_of_mention = int(file.read())
file.seek(0)
file.write(str(number_of_mention - 1))
file.close()
f.close()
else: # File not exist, create it and initialise it
log.debug("Sender file not exist")
f = open(sender_minute_filename,"w+")
f.write(str(1))
f.close()
can_continue = True
# use only notification
def on_notification(self, notification):
if can_continue:
id = notification['status']['id']
visibility = notification['status']['visibility']
if visibility == 'public':
visibility = 'unlisted'
mentions = notification['status']['mentions']
text = "@" + notification['status']["account"]["acct"] + " "
for mention in mentions:
if mention["acct"] != get_parameter("name_bot", config_file):
text = text + "@" + mention["acct"] + " "
if get_parameter("sensitive", config_file) == "yes":
sensitive = True
else:
sensitive = False
if self.args.source == "local":
media_dict = post_img_local(mastodon, get_parameter("default_text", config_file), log, config_file)
elif self.args.source == "distant":
media_dict = post_img_distant(mastodon, get_parameter("default_text", config_file), log, config_file)
mastodon.status_post(text, None, media_ids=[media_dict], sensitive=sensitive, visibility=visibility, spoiler_text=get_parameter("spoiler_text", config_file))
else:
log.debug("no picture send :(")
pass
else:
log.debug("Nevermind")
# catch only mention in notification
if notification['type'] == 'mention':
log.debug("Got a mention")
if notification["account"]["bot"] == False:
sender = notification['account']['acct'] # Get sender name
if sender in BLACKLIST:
log.info("Service refused to %s" % sender)
return
sender_hour_filename = "limiter/hour/" + sender; # Forge file for limiter
sender_minute_filename = "limiter/minute/" + sender; # Forge file for limiter
if os.path.isfile(sender_hour_filename): # Check if file exist
log.debug("Sender file exist")
statbuf = os.stat(sender_hour_filename)
last_edit = int(statbuf.st_mtime)
ts = int(time.time())
if ts - last_edit > 3599: # check if file is modified 1 hour after last edition
log.debug("file is too old")
f = open(sender_hour_filename,'w')
f.write(str(1)) # reset counter
f.close()
can_continue = True
else:
log.debug("file is young")
f = open(sender_hour_filename,'r+')
limit = int(get_parameter("limit_hour", config_file))
number_of_mention = int(f.read())
if number_of_mention < limit: # limit of mention per hour is limit_hour
log.debug("Sender have less of limit requests")
f.seek(0)
f.write(str(number_of_mention + 1))
can_continue = True
else:
log.debug("Sender have more of limit requests")
can_continue = False # if number of mention is for, user can't receive anything
f.close()
else: # File not exist, create it and initialise it
log.debug("Sender file not exist")
f = open(sender_hour_filename,"w+")
f.write(str(1))
f.close()
can_continue = True
if can_continue:
if os.path.isfile(sender_minute_filename): # Check if file exist
log.debug("Sender file exist")
statbuf = os.stat(sender_minute_filename)
last_edit = int(statbuf.st_mtime)
ts = int(time.time())
if ts - last_edit > 59: # check if file is modified 1 minute after last edition
log.debug("file is too old")
f = open(sender_minute_filename,'w')
f.write(str(1)) # reset counter
f.close()
can_continue = True
else:
log.debug("file is young")
f = open(sender_minute_filename,'r+')
limit = int(get_parameter("limit", config_file))
number_of_mention = int(f.read())
if number_of_mention < limit: # limit of mention per minute is 4
log.debug("Sender have less of limit requests")
f.seek(0)
f.write(str(number_of_mention + 1))
can_continue = True
else:
log.debug("Sender have more of limit requests")
can_continue = False # if number of mention is for, user can't receive anything
file = open(sender_hour_filename,'r+')
number_of_mention = int(file.read())
file.seek(0)
file.write(str(number_of_mention - 1))
file.close()
f.close()
else: # File not exist, create it and initialise it
log.debug("Sender file not exist")
f = open(sender_minute_filename,"w+")
f.write(str(1))
f.close()
can_continue = True
if can_continue:
id = notification['status']['id']
visibility = notification['status']['visibility']
if visibility == 'public':
visibility = 'unlisted'
mentions = notification['status']['mentions']
text = "@" + notification['status']["account"]["acct"] + " "
for mention in mentions:
if mention["acct"] != get_parameter("name_bot", config_file):
text = text + "@" + mention["acct"] + " "
if get_parameter("sensitive", config_file) == "yes":
sensitive = True
else:
sensitive = False
if self.args.source == "local":
media_dict = post_img_local(mastodon, get_parameter("default_text", config_file), log, config_file)
elif self.args.source == "distant":
media_dict = post_img_distant(mastodon, get_parameter("default_text", config_file), log, config_file)
elif self.args.source == "unsplash-random":
resp = post_unsplash_random_image(mastodon, log, config_file)
text = text + "\n" + resp['toot']
media_dict = resp['media_dict']
mastodon.status_post(text, id, media_ids=[media_dict], sensitive=sensitive, visibility=visibility, spoiler_text=get_parameter("spoiler_text", config_file))
else:
log.debug("no picture send :(")
pass
else:
log.debug("Nevermind")
def main():
parser = argparse.ArgumentParser(description='Choose between image or streaming')
parser.add_argument("-i", "--img", action='store_true', help="post image")
parser.add_argument("-s", "--source", help="Source of image [ local | distant ]")
parser.add_argument("--stream", action="store_true", help="stream user profile")
args = parser.parse_args()
if args.img:
if args.source == "local":
media_dict = post_img_local(mastodon, get_parameter("default_text", config_file), log, config_file)
elif args.source == "distant":
media_dict = post_img_distant(mastodon, get_parameter("default_text", config_file), log, config_file)
if get_parameter("sensitive", config_file) == "yes":
sensitive = True
else:
sensitive = False
mastodon.status_post(get_parameter("default_text", config_file), None, media_ids=[media_dict], sensitive=sensitive, visibility='public', spoiler_text=get_parameter("spoiler_text", config_file))
sys.exit()
elif args.stream:
stream = BotListener(args);
while True:
try:
log.info("Start listening...")
mastodon.stream_user(stream)
except Exception as error:
log.warning('General exception caught: ' + str(error))
time.sleep(0.5)
else:
print("Require an argument. Use --help to display help")
parser = argparse.ArgumentParser(description='Choose between image or streaming')
parser.add_argument("-i", "--img", action='store_true', help="post image")
parser.add_argument("-s", "--source", help="Source of image [ local | distant | unsplash-random ]")
parser.add_argument("--stream", action="store_true", help="stream user profile")
args = parser.parse_args()
if args.img:
text = get_parameter("default_text", config_file)
if args.source == "local":
media_dict = post_img_local(mastodon, get_parameter("default_text", config_file), log, config_file)
elif args.source == "distant":
media_dict = post_img_distant(mastodon, get_parameter("default_text", config_file), log, config_file)
elif args.source == "unsplash-random":
resp = post_unsplash_random_image(mastodon, log, config_file)
text = resp['toot']
media_dict = resp['media_dict']
if get_parameter("sensitive", config_file) == "yes":
sensitive = True
else:
sensitive = False
mastodon.status_post(text, None, media_ids=[media_dict], sensitive=sensitive, visibility='public', spoiler_text=get_parameter("spoiler_text", config_file))
sys.exit()
elif args.stream:
stream = BotListener(args);
while True:
try:
log.info("Start listening...")
mastodon.stream_user(stream)
except Exception as error:
log.warning('General exception caught: ' + str(error))
time.sleep(0.5)
else:
print("Require an argument. Use --help to display help")
main()

View file

@ -10,3 +10,4 @@ spoiler_text: some text here
limit: 2
limit_hour: 10
collection_url: https://source.unsplash.com/collection/<collection>/
unsplash_client_id: 03ad5bfbaa0acd6c96a728d425e533683ec25e5fb7fcf99f6461720b3d0d75a1

28
rename_all_file.sh Normal file
View file

@ -0,0 +1,28 @@
#!/bin/bash
function generate_random_char {
echo $( dd if=/dev/urandom bs=16 count=1|base64) > /tmp/rename_all_image
cp /tmp/rename_all_image /tmp/rename_all_image.back
sed -ie 's/[!@#\+\/$%^&*()=]//g' /tmp/rename_all_image.back
NEW_FILENAME=$(cat /tmp/rename_all_image.back)
EXTENSION=$(echo $img | cut -f 2 -d '.')
echo $NEW_FILENAME"."$EXTENSION
}
function move_file {
NEW_FILE=$(generate_random_char)
filepath=$2
IMG=$1
#echo $filepath"/"$NEW_FILE
if [ ! -f $filepath"/"$NEW_FILE ]; then
#mv $IMG $NEW_FILE
mv $IMG $filepath"/"$NEW_FILE
else
move_file $IMG
fi
}
for img in `ls $1/*`; do
filepath=$1
move_file $img $filepath
done