Copying img from HTML in Java Swing

Question

JTextPane text;
text.setText("somewords <img src=\"file:///C:/filepath/fire.png\" text=\"[fire1]\" title=\"[fire2]\" alt=\"[fire3]\" style=\"width:11px;height:11px;\"> otherwords");

Gives me this , which is as expected. But when I highlight it and copy paste it, I get "somewords otherwords". The same thing done inside Firefox when copied would paste "somewords [fire3] otherwords" (it substitutes alt text for image). Is there any way to replicate this behavior where the alt text is copied, or any other indication that a picture was copied? I'm guessing it is not a built in feature, so what I probably need to know is what should be overloaded to mimic this behavior.

Its for an output/chat window so its important that when the users quote it it includes the images (like emotes would)

Update: Successfully overrode the copyAction method... now what?

// (should) allow copying of alt text in place of images
class CustomEditorKit extends HTMLEditorKit {
    Action[] modifiedactions;
    CustomEditorKit() {
        int whereat=-1;
        modifiedactions=super.getActions();
        for(int k=0;k<super.getActions().length;k++) {
            if(super.getActions()[k] instanceof CopyAction) //find where they keep the copyaction
            {
                whereat=k;
                modifiedactions[whereat]=new CustomCopyAction(); //and replace it with a different one
            }
        }
    }
    @Override
    public Action[] getActions() {
        return modifiedactions; //returns the modified version instead of defaultActions
    }
    public static class CustomCopyAction extends TextAction {
        public CustomCopyAction() {
            super(copyAction); 
        }

        @Override
        public void actionPerformed(ActionEvent e) { //need to change this to substitute images with text, preferably their alt text.
            JTextComponent target = getTextComponent(e);
            //target.getText() gives full body of html, unbounded by selection area
            if (target != null) {
                target.copy(); //a confusing and seemingly never ending labyrinth of classes and methods
            }
        }
    }
}

Note: `import javafx.scene.web.WebEngine; import javafx.scene.web.WebView;` support this by default. — gunfulker, Oct 11 '15 at 16:13

score 2 · Answer 1 · answered Sep 28 '15 at 23:28

2

JTextPane provides method setEditorKit(EditorKit). I think you'll find your solution by providing a custom EditorKit.

You can override the copy and cut actions in a DefaultEditorKit, then pass it to JTextPane.

http://docs.oracle.com/javase/7/docs/api/javax/swing/text/DefaultEditorKit.html#copyAction

Or Java 8 introduces HTMLEditorKit that, if compatible with JTextPane, may provide the behavior you want.

https://docs.oracle.com/javase/8/docs/api/javax/swing/text/html/HTMLEditorKit.html

answered Sep 28 '15 at 23:28

Neal Ehardt

10,334
9
41
51

Thanks, I attempted that but ran into some complications. I'm attempting the first option you put forward. Firstly its a `StyledEditorKit` (easily done). Second there isn't a `CopyAction()`, instead its a class within `DefaultEditorKit` named that, its added to a list `Action[] defaultActions`, from which the actions are retrieved. So I tried `Overriding` this subclass. Didn't work. I also tried `getActions()[8]=new CustomCopyAction();` Didn't work. Neither of them ever enter the `actionPerformed()` method of the `TextAction` I wrote. Still trying things, any suggestions? – gunfulker Sep 29 '15 at 19:35
Extending the `HTMLEditorKit` instead makes the HTML render correctly, since it extends `StyledEditorKit`. `createInputAttributes` looks like it might take care of what I want? (3rd time editing this) – gunfulker Sep 29 '15 at 19:37
Success (on overriding the CustomCopyAction, still have to make it insert images text somehow) – gunfulker Sep 29 '15 at 22:19
Added what I've got so far, clueless on how to proceed. – gunfulker Sep 30 '15 at 06:03
I did some more digging, and I think @VGR's solution is much closer. I haven't run it, but it appears to have all the necessary parts (custom TransferHandler and DOM parsing). Good luck! – Neal Ehardt Sep 30 '15 at 19:29

VGR · Accepted Answer · 2015-10-01T13:01:53.480

The only way I can think of accomplishing this is by writing your own TransferHandler, and overriding the getSourceActions and exportToClipboard methods.

You can convert the HTML to plain text yourself, rather than letting Swing use the getSelectedText method of JTextPane, by recursively converting each Element of the HTML Document, customizing the conversion in the case where the Element has a NameAttribute of IMG and also has an ALT attribute.

Here's what I came up with:

import java.io.InputStream;
import java.io.ByteArrayInputStream;
import java.io.Reader;
import java.io.StringReader;
import java.io.StringWriter;
import java.io.IOException;

import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;

import java.util.Collection;
import java.util.Collections;
import java.util.LinkedHashSet;

import java.awt.EventQueue;
import java.awt.datatransfer.Clipboard;
import java.awt.datatransfer.DataFlavor;
import java.awt.datatransfer.Transferable;
import java.awt.datatransfer.UnsupportedFlavorException;

import javax.swing.JComponent;
import javax.swing.JFrame;
import javax.swing.JScrollPane;
import javax.swing.JTextPane;
import javax.swing.TransferHandler;

import javax.swing.text.AttributeSet;
import javax.swing.text.Document;
import javax.swing.text.Element;
import javax.swing.text.BadLocationException;
import javax.swing.text.html.HTML;

public class HTMLCopier
extends TransferHandler {
    private static final long serialVersionUID = 1;

    private final Collection<DataFlavor> flavors;

    HTMLCopier() {
        Collection<DataFlavor> flavorList = new LinkedHashSet<>();
        Collections.addAll(flavorList,
            new DataFlavor(String.class, null),
            DataFlavor.stringFlavor);

        String[] mimeTypes = {
            "text/html", "text/plain"
        };
        Class<?>[] textClasses = {
            Reader.class, String.class, CharBuffer.class, char[].class
        };
        Class<?>[] byteClasses = {
            InputStream.class, ByteBuffer.class, byte[].class
        };
        String[] charsets = {
            Charset.defaultCharset().name(),
            StandardCharsets.UTF_8.name(),
            StandardCharsets.UTF_16.name(),
            StandardCharsets.UTF_16BE.name(),
            StandardCharsets.UTF_16LE.name(),
            StandardCharsets.ISO_8859_1.name(),
            "windows-1252",
            StandardCharsets.US_ASCII.name(),
        };

        try {
            flavorList.add(new DataFlavor(
                DataFlavor.javaJVMLocalObjectMimeType +
                "; class=" + String.class.getName()));

            for (String mimeType : mimeTypes) {
                for (Class<?> textClass : textClasses) {
                    flavorList.add(new DataFlavor(String.format(
                        "%s; class=\"%s\"",
                        mimeType, textClass.getName())));
                }
                for (String charset : charsets) {
                    for (Class<?> byteClass : byteClasses) {
                        flavorList.add(new DataFlavor(String.format(
                            "%s; charset=%s; class=\"%s\"",
                            mimeType, charset, byteClass.getName())));
                    }
                }
            }

            for (String mimeType : mimeTypes) {
                flavorList.add(new DataFlavor(String.format(
                    "%s; charset=unicode; class=\"%s\"",
                    mimeType, InputStream.class.getName())));
            }
        } catch (ClassNotFoundException e) {
            throw new RuntimeException(e);
        }

        this.flavors = Collections.unmodifiableCollection(flavorList);
    }

    @Override
    public int getSourceActions(JComponent component) {
        return COPY_OR_MOVE;
    }

    @Override
    public void exportToClipboard(JComponent component,
                                  Clipboard clipboard,
                                  int action) {
        JTextPane pane = (JTextPane) component;
        Document doc = pane.getDocument();

        int start = pane.getSelectionStart();
        int end = pane.getSelectionEnd();

        final String html;
        final String plainText;
        try {
            StringWriter writer = new StringWriter(end - start);
            pane.getEditorKit().write(writer, doc, start, end - start);
            html = writer.toString();

            StringBuilder plainTextBuilder = new StringBuilder();
            appendTextContent(doc.getDefaultRootElement(), start, end,
                plainTextBuilder);
            plainText = plainTextBuilder.toString();
        } catch (BadLocationException | IOException e) {
            throw new RuntimeException(e);
        }

        Transferable contents = new Transferable() {
            @Override
            public boolean isDataFlavorSupported(DataFlavor flavor) {
                return flavors.contains(flavor);
            }

            @Override
            public DataFlavor[] getTransferDataFlavors() {
                return flavors.toArray(new DataFlavor[0]);
            }

            @Override
            public Object getTransferData(DataFlavor flavor)
            throws UnsupportedFlavorException,
                   IOException {

                String data;
                if (flavor.isMimeTypeEqual("text/html")) {
                    data = html;
                } else {
                    data = plainText;
                }

                Class<?> dataClass = flavor.getRepresentationClass();
                if (dataClass.equals(char[].class)) {
                    return data.toCharArray();
                }
                if (flavor.isRepresentationClassReader()) {
                    return new StringReader(data);
                }
                if (flavor.isRepresentationClassCharBuffer()) {
                    return CharBuffer.wrap(data);
                }
                if (flavor.isRepresentationClassByteBuffer()) {
                    String charset = flavor.getParameter("charset");
                    return Charset.forName(charset).encode(data);
                }
                if (flavor.isRepresentationClassInputStream()) {
                    String charset = flavor.getParameter("charset");
                    return new ByteArrayInputStream(
                        data.getBytes(charset));
                }
                if (dataClass.equals(byte[].class)) {
                    String charset = flavor.getParameter("charset");
                    return data.getBytes(charset);
                }
                return data;
            }
        };

        clipboard.setContents(contents, null);

        if (action == MOVE) {
            pane.replaceSelection("");
        }
    }

    private void appendTextContent(Element element,
                                   int textStart,
                                   int textEnd,
                                   StringBuilder content)
    throws BadLocationException {
        int start = element.getStartOffset();
        int end = element.getEndOffset();
        if (end < textStart || start >= textEnd) {
            return;
        }

        start = Math.max(start, textStart);
        end = Math.min(end, textEnd);

        AttributeSet attr = element.getAttributes();
        Object tag = attr.getAttribute(AttributeSet.NameAttribute);

        if (tag.equals(HTML.Tag.HEAD) ||
            tag.equals(HTML.Tag.TITLE) ||
            tag.equals(HTML.Tag.COMMENT) ||
            tag.equals(HTML.Tag.SCRIPT)) {

            return;
        }

        if (tag.equals(HTML.Tag.INPUT) ||
            tag.equals(HTML.Tag.TEXTAREA) ||
            tag.equals(HTML.Tag.SELECT)) {

            // Swing doesn't provide a way to read input values
            // dynamically (as far as I know;  I could be wrong).
            return;
        }

        if (tag.equals(HTML.Tag.IMG)) {
            Object altText = attr.getAttribute(HTML.Attribute.ALT);
            if (altText != null) {
                content.append(altText);
            }
            return;
        }

        if (tag.equals(HTML.Tag.CONTENT)) {
            content.append(
                element.getDocument().getText(start, end - start));
            return;
        }

        int count = element.getElementCount();
        for (int i = 0; i < count; i++) {
            appendTextContent(element.getElement(i), textStart, textEnd,
                content);
        }
    }

    public static void main(String[] args) {
        EventQueue.invokeLater(new Runnable() {
            @Override
            public void run() {
                JTextPane text = new JTextPane();
                text.setContentType("text/html");
                text.setEditable(false);
                text.setText("somewords <img src=\"file:///C:/filepath/fire.png\" text=\"[fire1]\" title=\"[fire2]\" alt=\"[fire3]\" style=\"width:11px;height:11px;\"> otherwords");

                text.setTransferHandler(new HTMLCopier());

                JFrame window = new JFrame("HTML Copier");
                window.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
                window.getContentPane().add(new JScrollPane(text));
                window.pack();
                window.setLocationByPlatform(true);
                window.setVisible(true);

                text.selectAll();
                text.copy();
            }
        });
    }
}

Edit: Updated code to properly place only highlighted text on clipboard.

Oh, it copies the whole thing every time, regardless of what is highlighted. Figuring that out now... — gunfulker, Oct 01 '15 at 04:11

Copying img from HTML in Java Swing

2 Answers2