66<a href =" https://github.com/dotcommander/defuddle/releases " ><img src =" https://img.shields.io/github/v/release/dotcommander/defuddle " alt =" Release " ></a >
77<a href =" https://github.com/dotcommander/defuddle/actions " ><img src =" https://github.com/dotcommander/defuddle/workflows/Test/badge.svg " alt =" Tests " ></a >
88<a href =" https://goreportcard.com/report/github.com/dotcommander/defuddle " ><img src =" https://goreportcard.com/badge/github.com/dotcommander/defuddle " alt =" Go Report Card " ></a >
9- <a href =" https://godoc.org /github.com/dotcommander /defuddle " ><img src =" https://godoc.org/ github.com/dotcommander /defuddle?status .svg " alt =" GoDoc " ></a >
9+ <a href =" https://pkg.go.dev /github.com/kaptinlin /defuddle-go " ><img src =" https://pkg.go.dev/badge/ github.com/kaptinlin /defuddle-go .svg " alt =" Go Reference " ></a >
1010</p >
1111
1212## Introduction
@@ -22,15 +22,15 @@ Available as both a **Go library** and a drop-in **CLI tool** compatible with th
2222Download a pre-built binary from the [ releases page] ( https://github.com/dotcommander/defuddle/releases ) , or install with Go:
2323
2424``` bash
25- go install github.com/dotcommander /defuddle/cmd/defuddle@latest
25+ go install github.com/kaptinlin /defuddle-go /cmd/defuddle@latest
2626```
2727
2828### Library
2929
3030Require Defuddle Go using ` go get ` :
3131
3232``` bash
33- go get github.com/dotcommander /defuddle
33+ go get github.com/kaptinlin /defuddle-go
3434```
3535
3636> Requires Go 1.26 or higher.
@@ -133,6 +133,7 @@ Defuddle automatically detects popular platforms and applies specialized extract
133133| Reddit | Posts with comment trees |
134134| Substack | Newsletter articles |
135135| Twitter / X | Tweets and threads |
136+ | X Articles | Long-form articles (Draft.js) |
136137| YouTube | Video metadata and descriptions |
137138
138139### Custom Extractors
@@ -141,30 +142,32 @@ Implement the `BaseExtractor` interface to add support for any site:
141142
142143``` go
143144type MyExtractor struct {
144- doc *goquery.Document
145- url string
146- schemaOrgData any
145+ *extractors.ExtractorBase
147146}
148147
149- func (e *MyExtractor ) Name () string { return " MyExtractor" }
148+ func NewMyExtractor (doc *goquery .Document , url string , schema any ) extractors .BaseExtractor {
149+ return &MyExtractor{ExtractorBase: extractors.NewExtractorBase (doc, url, schema)}
150+ }
151+
152+ func (e *MyExtractor ) Name () string { return " MyExtractor" }
150153func (e *MyExtractor ) CanExtract () bool { return true }
151154
152- func (e *MyExtractor ) Extract () (* defuddle . ExtractedContent , error ) {
153- title := e.doc . Find ( " h1.article-title " ). Text ()
154- content , _ := e. doc .Find (" .article-body" ).Html ()
155- return &defuddle. ExtractedContent {
156- Title: &title ,
157- ContentHTML: &content ,
158- }, nil
155+ func (e *MyExtractor ) Extract () * extractors . ExtractorResult {
156+ doc := e.GetDocument ()
157+ content , _ := doc.Find (" .article-body" ).Html ()
158+ return &extractors. ExtractorResult {
159+ ContentHTML: content ,
160+ Variables: map [ string ] string { " site " : " My Site " } ,
161+ }
159162}
160163```
161164
162165Register it before parsing:
163166
164167``` go
165- extractors.DefaultRegistry . Register (extractors.ExtractorMapping {
168+ extractors.Register (extractors.ExtractorMapping {
166169 Patterns : []any{" mysite.com" },
167- Extractor : func (doc, url, schema) { return &MyExtractor{doc, url, schema} } ,
170+ Extractor : NewMyExtractor ,
168171})
169172```
170173
0 commit comments