You are not logged in.

kio slave

Dear visitor, welcome to KDE-Forum.org. If this is your first visit here, please read the Help. It explains in detail how this page works. To use all features of this page, you should consider registering. Please use the registration form, to register here or read more information about the registration process. If you are already registered, please login here.

boyadg

Unregistered

1

Saturday, April 24th 2004, 5:16pm

kio slave

What I want to do:

I am trying to create a kio_slave that spoofs (forges) the referer when a page is requested. I want to specify the referer in the URL, and for this I chose to use the reference part of the URL.

Something like this:
spoof://www.page.com#http://www.spoofedreferer.com
should request http://www.page.com but using the (spoofed) referer http://www.spoofedreferer.com


What I already did:

I read the tutorial at http://www.heise.de/ct/english/01/05/242/ (I wish it wasn't so short) and got a basic kio_spoof slave working. I made my own "get" function that parses the url and separates it into a spoof_url and a spoof_referer. Then, like in the tutorial, it shows these values in a handmade webpage.
After that I searched for ideas on how to actually open the spoof_url using the spoof_referer but after reading the KDE API reference over and over again, the only place I found the referer value is in the HTTPProtocol code (kio_http). That's when I tried copying all of the kio_http code and start from there, but whatever I did, I couldn't even get that to compile.


What I need:

I just need some help to get on the way. How do I "GET" www.page.com (using my own referer)? Do I have to write the code for the entire process of getting the page (like the kio_http code), or can I somehow use the HTTPProcotol class?

I'm just starting with C++, so if someone would be so kind to get me on the right track, please be as detailed as possible.

Thanks in advance for your attention.

dfaure

Beginner

Posts: 16

Location: France

  • Send private message

2

Sunday, April 25th 2004, 1:30pm

RE: kio slave

Instead of writing a whole new kioslave, how about setting the referrer metadata and using http:// ?

The metadata key is simply "referrer" (see kdelibs/kio/DESIGN.metadata)

It's quite difficult to reuse an existing slave's code from another one...
David Faure, KDE developer, Klaralvdalens-Datakonsult AB.

boyadg

Unregistered

3

Thursday, May 6th 2004, 4:24pm

Sorry for the late response, I had more or less given up on the thing.

Thanks a lot for the tip dfaure, but I can't seem to get it to work! Here's what I tried:

KIO::TransferJob *tjob = KIO::get( spoofUrl, false, false );
tjob->addMetaData( "referrer", spoofReferer.url() );
QObject::connect(
tjob, SIGNAL( data( KIO::Job*, const QByteArray& ) ),
this, SLOT( slotData( KIO::Job*, const QByteArray& ) )
);

but that returns an error while trying to compile:

no matching function for call to
QObject::connect( tjob, SIGNAL( data( KIO::Job*, const QByteArray& ) ), this, SLOT( slotData( KIO::Job*, const QByteArray& ) ) );
candidates are:
static bool QObject::connect(const QObject*, const char*, const QObject*, const char*)
bool QObject::connect(const QObject*, const char*, const char*) const

I don't understand, am I not using the first candidate (object, signal, object, slot)?

And something else. Is it possible to not even have to create the slotData function, and just connect the data signal to the data output function. And if it isn't possible, what should the slotData function look like? Something like this?

void kio_spoofProtocol::slotData( KIO::Job *job, const QByteArray &data )
{
data( data );
finished();
}


I hope someone can help me again. Thanks in advance.

dfaure

Beginner

Posts: 16

Location: France

  • Send private message

4

Thursday, May 6th 2004, 4:50pm

I guess the application is not one you're writing yourself, but e.g. Konqueror, right?

So this code is all inside the slave?
In that case you have to
1) make sure you create a KApplication, not just a KInstance, in the slave (this is necessary for using other slaves, like kio_audiocd does)
2) create a QObject (I guess the code doesn't compile because "this" isn't a QObject). The simplest is probably to multiply inherit with "public QObject, public SlaveBase".
3) forward the data indeed, but don't call finished() there! slotData will be called more than once. Instead, connect to the job's result(KIO::Job*) signal and call finished() from there.
David Faure, KDE developer, Klaralvdalens-Datakonsult AB.

boyadg

Unregistered

5

Thursday, May 6th 2004, 9:22pm

Thanks again dfaure for your help. I feel I'm so close, but still not there. I can see now that my idea requires a lot more knowledge about how the KDE API works than I thought.

I am using a kio-slave template autogenerated by KDevelop and you are right about the fact that it uses KInstance. I changed that to:
KApplication app( argc, argv, "kio_spoof", false, false );

Your tip about the QObject also worked perfectly. My mistake was that I actually thought that all KDE classes were derived from QObject.

I connected data to slotData, and result to slotResult:
QObject::connect( tjob, SIGNAL( data( KIO::Job *, const QByteArray & ) ), this, SLOT( slotData( KIO::Job *, const QByteArray & ) ) );
QObject::connect( tjob, SIGNAL( result( KIO::Job * ) ), this, SLOT( slotResult( KIO::Job * ) ) );

and this is what I put in the "slot-functions":
void kio_spoofProtocol::slotData( KIO::Job *job, const QByteArray &data )
{ kio_spoofProtocol::data( data ); }
and
void kio_spoofProtocol::slotResult( KIO::Job *job )
{ finished(); }

All of this compiles without errors, just 2 warnings about
"unused parameter KIO::Job*job"
and 1 about
"__comp_ctor is deprecated (declared at kapplication.h:172)"

Here's where I got really happy, but sadly, when I install and try it, it doesn't load any page, nor does it output any error. It just sits there as if it's loading, but without changing the page.

Is there any obvious error in my code?
I almost hope so, because I read that kio-slaves are hard to debug.

It would be so incredibly cool if I got this to work. I and a lot of friends would be very happy to have a working referer "spoof-slave".

dfaure

Beginner

Posts: 16

Location: France

  • Send private message

6

Thursday, May 6th 2004, 9:58pm

I guess the problem is the rest of your get() implementation... or a setup problem (kioslave not launched at all)....
You do need to debug it, e.g. adding debug output to it... To see its output, simply type "kdeinit" in a terminal, further kioslaves will be forked from there so their output will be in that terminal.

If you want to use gdb, see http://webcvs.kde.org/cgi-bin/cvsweb.cgi…kup&sortby=date
David Faure, KDE developer, Klaralvdalens-Datakonsult AB.

boyadg

Unregistered

7

Thursday, May 6th 2004, 11:34pm

You are again absolutely right. I first reduced the get() function to the absolute minimum, it now only contains:

KIO::TransferJob *tjob = KIO::get( KURL("http://www.google.com"), false, false );

and

QObject::connect( tjob, SIGNAL( data( KIO::Job *, const QByteArray & ) ), this, SLOT( slotData( KIO::Job *, const QByteArray & ) ) );
QObject::connect( tjob, SIGNAL( result( KIO::Job * ) ), this, SLOT( slotResult( KIO::Job * ) ) );

then, while debugging, I got this output:

kio_file: *** Starting kio_spoof
kio_file: kio_spoofProtocol::kio_spoofProtocol()
kio_file: kio_spoofProtocol::get()
QObject::connect: No such slot QObject::slotData(KIO::Job*,const QByteArray&)
QObject::connect: (sender name: 'job')
QObject::connect: (receiver name: 'unnamed')
QObject::connect: No such slot QObject::slotResult(KIO::Job*)
QObject::connect: (sender name: 'job')
QObject::connect: (receiver name: 'unnamed')

I believe that what's wrong is that slotData isn't a funcion of QObject. So I tried to change it to kio_spoofProtocol::slotResult (in the connect lines), but that didn't work. I also tried to remove QObject in front of connect, but then it seems to get mixed up with some other connect funcion.
Maybe I should change the "this" words in the connect lines? I will investigate further tomorrow.

In the meantime, thanks dfaure, you've been of invaluable help to me. If you're not fed up with me, please visit this post some more, cause I'll be asking until I get this working, or until nobody responds anymore.

Goodnight!

dfaure

Beginner

Posts: 16

Location: France

  • Send private message

8

Friday, May 7th 2004, 6:40am

You need to define the slots as slots.
In the .h file, put the Q_OBJECT macro at the right place, like
class MyProtocol : public QObject
{
Q_OBJECT
public:
...

and later on define slots:

public slots:
void slotData( ... );
void slotResult( ... );

Then ensure that "moc" runs over that file (get kdevelop to regenerate the Makefiles or type "make force-reedit"), and include the .moc from the .cpp

This is standard QObject/moc stuff, and is necessary in order to define slots.
David Faure, KDE developer, Klaralvdalens-Datakonsult AB.

boyadg

Unregistered

9

Friday, May 7th 2004, 10:17am

A little closer. I did define the slots as slots, but I didn't add the Q_OBJECT macro. After it, I had to fight for a while with KDevelop to make it use the moc, but I finally got it to compile again, without errors.

I also added two more slots, for the connected() and the canceled() signals.

Finally, I added a debug message at the end of the get() function.

Now when I try the slave, I get this debug output:
kio_file: *** Starting kio_spoof
kio_file: kio_spoofProtocol::kio_spoofProtocol()
kio_file: kio_spoofProtocol::get()
kio_file: END of kio_spoofProtocol::get()

As you can see, it never even calls the slotData and slotResult functions. You can see that in Konqueror too, because it just loads and loads, without ever showing anything, not even an error.

After clicking the Stop button in Konqueror, I get:
kio_file: *** kio_spoof Done
kio_file: kio_spoofProtocol::~kio_spoofProtocol()
kio (KLauncher): ERROR: SlavePool: No communication with slave.

Finally, and forgive me if this is a terrible newbie question, I noticed that the end of the get() function is reached. Doesn't this cause the KIO::TransferJob *tjob variable to get deleted?
Shouldn't I declare KIO::TransferJob *tjob as a member variable of the class (in the .h file)? Or maybe even use new and delete?


I'm learning a lot, but I hate not being able to find this stuff out for myself... and the slave still doesn't work.

dfaure

Beginner

Posts: 16

Location: France

  • Send private message

10

Friday, May 7th 2004, 10:25am

KIO::get() uses new to create the job, no problem there.
It's async, so it's normal that it returns immediately, and that your get() returns early.
But when the data comes it should appear there..... I'm not sure what else you do in get(), but I don't see what the problem is right now.
David Faure, KDE developer, Klaralvdalens-Datakonsult AB.

boyadg

Unregistered

11

Friday, May 7th 2004, 12:04pm

> I'm not sure what else you do in get()...

I reduced it to a minimum, with a hardcoded url and no referer. It really only contains this now:

void kio_spoofProtocol::get(const KURL& url )
{

kdDebug(7101) << "kio_spoofProtocol::get()" << endl;
KIO::TransferJob *tjob = KIO::get( KURL("http://www.google.com"), false, false );

if( !connect( tjob, SIGNAL( data( KIO::Job *, const QByteArray & ) ), this, SLOT( slotData( KIO::Job *, const QByteArray & ) ) ) )
kdDebug(7101) << "connect data to slotData returned false" << endl;
else
kdDebug(7101) << "Successfully connected data to slotData" << endl;

if( !connect( tjob, SIGNAL( result( KIO::Job * ) ), this, SLOT( slotResult( KIO::Job * ) ) ) )
kdDebug(7101) << "connect result to slotResult returned false" << endl;
else
kdDebug(7101) << "Successfully connected result to slotResult" << endl;
}

as you can see, I check if connect() returns true, which it does.

Could it be my spoof.protocol file? Right now, it contains:
[Protocol]
exec=kio_spoof
protocol=spoof
input=none
output=stream
reading=true
defaultMimetype=text/html
Icon=remote
Description=A kioslave for spoof

cristianbdg

Beginner

Posts: 6

  • Send private message

12

Tuesday, October 26th 2004, 9:36pm

new try

after having learned a lot of new things about programming, I decided to go back to my frustrated project, just to see if the new things I learned would help me to get this working. I actually started all over again, but after two days I find myself with practically the same code as before, but now it works even worse. The problem is that the what I previously used to create a KApplication now has a KDE_DEPRECATED next to it in the documentation (and it says it will make my app crash). I tried to do something with KCmdLineArgs and KAboutData, but I get all kinds of error messages.

So now I'm even further from the solution than before, because I have to get the KApplication part right and then get the slave to work.

I really don't get it, because the code is at its absolute minimum. I don't see why it wouldn't work. Here's the complete code, just without includes, comments and debug messages:

spoof.cpp

Source code

 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

using namespace KIO;

extern "C"
{
   int kdemain( int argc, char** argv )
   {
      // This causes an error, see below the code
      KApplication app();
      if( argc != 4 )
         exit( -1 );
      kio_spoofProtocol slave( argv[2], argv[3] );
      slave.dispatchLoop();
      return 0;
   }
}

kio_spoofProtocol::kio_spoofProtocol( const QCString &pool_socket, const QCString &app_socket )
   : SlaveBase( "kio_spoof", pool_socket, app_socket )
{}

kio_spoofProtocol::~kio_spoofProtocol()
{}

void kio_spoofProtocol::get( const KURL& url )
{
   KIO::TransferJob* getJob = KIO::get( KURL( "http://www.google.com" ), false, false );
   connect( getJob, SIGNAL( data( KIO::Job*, const QByteArray& ) ), this, SLOT( slotData( KIO::Job*, const QByteArray& ) ) );
   connect( getJob, SIGNAL( result( KIO::Job* ) ), this, SLOT( slotResult( KIO::Job* ) ) );
}

void kio_spoofProtocol::slotData( KIO::Job* job, const QByteArray& data )
{
   kio_spoofProtocol::data( data );
}

void kio_spoofProtocol::slotResult( KIO::Job* job )
{
   finished();
}

void kio_spoofProtocol::mimetype( const KURL& url )
{
   mimeType( "text/html" );
   finished();
}

#include "spoof.moc"


spoof.h

Source code

 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

class QCString;

class kio_spoofProtocol : public QObject, public KIO::SlaveBase
{
   Q_OBJECT

public:
   kio_spoofProtocol( const QCString& pool_socket, const QCString& app_socket );
   virtual ~kio_spoofProtocol();
   virtual void mimetype( const KURL& url );
   virtual void get( const KURL& url );

public slots:
   void slotData( KIO::Job* job, const QByteArray& data );
   void slotResult( KIO::Job* job );

};


spoof.protocol

Source code

 
1
2
3
4
5
6
7
8
9

[Protocol]
exec=kio_spoof
protocol=spoof
input=none
output=stream
reading=true
defaultMimetype=text/html
Icon=remote
Description=A kioslave to spoof the referer


Just doing a KApplication app causes this error message:
"FAILURE (KCmdLineArgs): Application has not called KCmdLineArgs::init(...).

Adding a KCmdLineArgs::init() means adding a KAboutData so I tried:
KAboutData about( "kio_spoof", I18N_NOOP( "kio_spoof" ), "0" );
KCmdLineArgs::init( argc, argv, &about );

which gets me a new error message
kio_spoof: Unexpected argument 'spoof'.

I'm completely lost, so any tips about the code are welcome. I'm sorry for putting all the code in the message, but I thought that might be clearer. Thanks in advance.

cristianbdg

Beginner

Posts: 6

  • Send private message

13

Wednesday, October 27th 2004, 12:41pm

Sorry to be talking all alone, but I've got some new findings:

1. I saw that the audiocd slave still uses
KApplication app( argc, argv, "kio_spoof", false, true );
so I put that in my slave too. It generates a warning:
warning: `__comp_ctor' is deprecated (declared at /usr/kde/3.3/include/kapplication.h:198)
but other than that the slave starts up ok, so I'm leaving it to be able to concentrate on the other problem.

2. Putting debug messages in the code proves that
kio_spoofProtocol::get( const KURL& url )
runs all the way to the end, so KIO::get gets executed

3. Now only errormessages appear when hitting reload, so I don't know if they are important:
kio (KIOConnection): ERROR: Header read failed, errno=104
kio (KIOConnection): ERROR: Header has invalid size (-1)
kio (KLauncher): ERROR: SlavePool: No communication with slave.

4. I hooked up almost all signals that a Transferjob (the one returned by KIO::get) emits to slots and not a single one gets called, not even result()!!! How can this be?

Could it be that the KIO::get job gets destroyed somehow?

anda_skoa

Professional

Posts: 1,273

Location: Graz, Austria

Occupation: Software Developer

  • Send private message

14

Thursday, October 28th 2004, 5:37pm

Do you call the KApplication's exec() method somewhere, i.e. do you start the event loop?

Cheers,
_
Qt/KDE Developer
Debian User

cristianbdg

Beginner

Posts: 6

  • Send private message

15

Thursday, October 28th 2004, 8:49pm

> "Do you call the KApplication's exec() method somewhere, i.e. do you start the event loop?"

Well, no, I didn't know I had to. (But I can understand it's necessary).

I changed "return 0;" in kdemain to "return app.exec();" and immediately things changed, although unfortunately it still doesn't work.

Now the error I get is:
kio (KLauncher): ERROR: SlavePool: Unexpected data from slave.
The error occurs after slotResult() finishes.

The strange part is that slotData still doesn't get called until I click Go for the second time, and at the same time it (slotData) sometimes gets called more than once after click Stop.

Here's the debug output, I hope you don't mind putting it all here:

Source code

 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

I write spoof://www.whatever.com and click on Go

kio_file: *** kio_spoof Starting
kio_spoofProtocol Constructor
get() url: spoof://www.whatever.com
get() mimetype: text/html
get() getJob created
get() end

nothing more happens so here I click Go again

kio_file: *** kio_spoof Done
kio_file: *** kio_spoof Starting
kio_spoofProtocol Constructor
get() url: spoof://www.whatever.com
get() mimetype: text/html
get() getJob created
get() end
slotData() data: [lots of data ...]
slotData() data: [more data ...]
slotData() data: []
slotResult() getJob finished without errors
kio (KLauncher): ERROR: SlavePool: Unexpected data from slave.

finally here I click Stop

kio_file: *** kio_spoof Done
slotData() data: [the same lots of data as before ...]
slotData() end
slotData() data: [the same more data as befor ...]
slotData() end
slotData() data: []
slotData() end
slotResult() getJob finished without errors
slotResult() end
kio (KLauncher): ERROR: SlavePool: Unexpected data from slave.


I hope that someone understands what the (new) problem is. I just don't have enough experience with the KDE API to find mistakes like the app.exec() one. Even so, I would really, really like to get this to work.

Thanks again for your attention.

cristianbdg

Beginner

Posts: 6

  • Send private message

16

Saturday, October 30th 2004, 12:12pm

I finally decided to do it completely differently. I just downloaded the source of the http kio-slave, changed the names and some lines of the code so that it uses the reference part of the url as the referer and not the normal one and now it works. It still has a few quirks, but I'm pretty sure that I can solve them by taking a closer look at the code.

Thanks anyway for your help!