[booru] Change Directory name and add Blacklist in config.json

[alfa-147]

Hi everyone!,
I'm currently downloading with the configuration:
"directory": ["{tags_copyright[N]}"],
so the posts have their own directory easy to recognize.
The situation, now, is that Gallery-dl can creates a lot of directory this way like:

"pokemon_(game)"
"pokemon_sun_&_moon"
"pokemon_(anime)"
"pokemon_the_series_sun_&_moon" 

all folders that clearly can be under the name "pokemon".

My first question:
Is there a way to replace the value obtained from {tags copyright[N]} with one I decide, by creating a list of replacement or something?
For example: value obtained >>"mori_calliope" changed to value replaced >>"hololive".

My second question:
Is possible to create a Blacklist for some tags(general, metadata...)?

[mikf]

My first question:

It is possible, but quite involved and will require creating a new .py file and writing some Python code, i.e. this is not natively supported. I'll post an example using \fM format strings tomorrow if you are interested.

My second question:

Use image-filter in a config file or --filter on CLI and use, for example, the contains function or regular expressions.

--filter "not contains(tags, ('tag1', 'tag2', 'tag3'))"

[mikf]

{
    "extractor": {
        "directory": ["\fM /tmp/gdl_booru.py:replace_copyright", "{tags_copyright[0]}"]
    }
}

REPLACEMENTS = {
    "pokemon_(game)"               : "pokemon",
    "pokemon_sun_&_moon"           : "pokemon",
    "pokemon_(anime)"              : "pokemon",
    "pokemon_the_series_sun_&_moon": "pokemon",
    "mori_calliope"                : "hololive",
}


def replace_copyright(metadata):
    copyright = metadata.get("tags_copyright")
    if not copyright:
        return ""
    return REPLACEMENTS.get(copyright[0], "")

[alfa-147]

Thanks for your reply, I hope this doesn't bother you too much.
I was wondering if this can be applied even to subdirectory and filenames using the right format.

[mikf]

Probably. It would be a lot easier if I had a concrete example, i.e. a URL + how its folder structure and names are supposed to look like.

[alfa-147]

Sure.
My intention is to structure the folders to have the principal copyright as main directory and secondary copyright (if exist) as its subdirectory.
Let's say this Example: https://danbooru.donmai.us/posts/8183070 (SFW)
The folder's structure will be:
...\Genshin_Impact\Hololive_english\md5-jon eve.jpg
What i'm searching for is:
...\𝗚𝗘𝗡𝗦𝗛𝗜𝗡 𝗜𝗠𝗣𝗔𝗖𝗧\𝗛𝗢𝗟𝗢𝗟𝗜𝗩𝗘\md5-𝘫𝘰𝘯 𝘦𝘷𝘦.jpg where the copyrights are replaced with unicode characters, same for the artist.

[mikf]

config.json

{
    "extractor": {
        "danbooru": {
            "directory": ["\fM /tmp/gdl_booru.py:replace", "{tags_copyright[0]:?//}", "{tags_copyright[1]:?//}"],
            "filename": "{md5}{tags_artist[0]:?-//}.{extension}"
        }
    }
}

gdl_booru.py

TR = str.maketrans(
    "_abcdefghijklmnopqrstuvwxyz",
    " 𝗔𝗕𝗖𝗗𝗘𝗙𝗚𝗛𝗜𝗝𝗞𝗟𝗠𝗡𝗢𝗣𝗤𝗥𝗦𝗧𝗨𝗩𝗪𝗫𝗬𝗭",
)

def replace(metadata):
    for key in (
        "tags_artist",
        "tags_copyright",
    ):
        tags = metadata.get(key)
        if not tags:
            continue
        for idx, tag in enumerate(tags):
            tags[idx] = tag.translate(TR)

    return ""

This replaces all tag values in tags_copyright and tags_artist with bold Unicode characters.

$ gallery-dl https://danbooru.donmai.us/posts/8183070
./𝗚𝗘𝗡𝗦𝗛𝗜𝗡 𝗜𝗠𝗣𝗔𝗖𝗧/𝗛𝗢𝗟𝗢𝗟𝗜𝗩𝗘/8cc2845623322ab664e6a9acb2b58223-𝗝𝗢𝗡 𝗘𝗩𝗘.jpg

[alfa-147]

Thanks again for all the support.

[alfa-147]

Sorry to bother you again but I can't seem to solve this problem myself.
I'm actually trying to combine the two Python codes together so that I have REPLACEMENTS = {} for tags_copyright which is much preferable and str.maketrans() for tags_artist which handles the names automatically. My problem is that while the str.maketrans() part works fine, the REPLACEMENTS = {} part seems to do nothing (failed on Danbooru, Sankaku...) and I don't understand why.
Here a copy of the Python file:

REPLACEMENTS = {
    "azur lane"                : "𝐀𝐙𝐔𝐑 𝐋𝐀𝐍𝐄",
}


def replace_copyright(metadata):
    copyright = metadata.get("tags_copyright")
    if not copyright:
        return ""
    return REPLACEMENTS.get(copyright[0], "")

TR = str.maketrans(
    "abcdefghijklmnopqrstuvwxyz0123456789",
    "𝖠𝖡𝖢𝖣𝖤𝖥𝖦𝖧𝖨𝖩𝖪𝖫𝖬𝖭𝖮𝖯𝖰𝖱𝖲𝖳𝖴𝖵𝖶𝖷𝖸𝖹𝟢𝟣𝟤𝟥𝟦𝟧𝟨𝟩𝟪𝟫",
)

def replace(metadata):
    for key in (
        "tags_artist",
    ):
        tags = metadata.get(key)
        if not tags:
            continue
        for idx, tag in enumerate(tags):
            tags[idx] = tag.translate(TR)
    return ""

Thanks in advance for any advice.

[mikf]

You need to replace the list elements of tags_copyright in-place and actually have it call the replace_copyright() function for it to have an effect. Try the following with the config settings from #6845 (comment) :

REPLACEMENTS = {
    "azur lane"                : "𝐀𝐙𝐔𝐑 𝐋𝐀𝐍𝐄",
}

TR = str.maketrans(
    "abcdefghijklmnopqrstuvwxyz0123456789",
    "𝖠𝖡𝖢𝖣𝖤𝖥𝖦𝖧𝖨𝖩𝖪𝖫𝖬𝖭𝖮𝖯𝖰𝖱𝖲𝖳𝖴𝖵𝖶𝖷𝖸𝖹𝟢𝟣𝟤𝟥𝟦𝟧𝟨𝟩𝟪𝟫",
)


def replace_copyright(metadata):
    tags = metadata.get("tags_copyright")
    if tags:
        for idx, tag in enumerate(tags):
            tags[idx] = REPLACEMENTS.get(tag,tag)


def replace_artist(metadata):
    tags = metadata.get("tags_artist")
    if tags:
        for idx, tag in enumerate(tags):
            tags[idx] = tag.translate(TR)


def replace(metadata):
    replace_artist(metadata)
    replace_copyright(metadata)
    return ""

---

"azur lane"

The actual tag is azur_lane with an underscore, I think.