Pre-release test
From Ekiga
Contents |
Ekiga pre-release test flow
Having bugs is not the problem. That happens to all software. But a bug that's something obvious and that just wasn't fixed because nobody tested some simple use-cases, is bad. To minimize that risk in future, we should follow a basic test-flow.
Basic source code review
- configure[.in] checks for the right versions of PTLib/OPAL
- configure script was generated
Compile time
For these tests, please compile with LC_ALL=C set in environment.
- make >/dev/null - any error messages?
Runtime
For all runtime tests, please run Ekiga with LC_ALL=C set in environment. That ensures the strings specified in the code are used, and the error messages tell us something.
The basic structure of the runtime tests follows the categories describing the features of Ekiga.
GUI general
- Obvious HIG violations?
- Typos in strings?
- Something written unclear? Everything makes sense?
- Do all hotkeys work?
- Breaking behaviour of popup-windows?
- Does the tray icon work like intended?
Documentation
Verify the documentation follows recommendations from the GNOME Documentation Style Guide and the GNOME Handbook of Writing Software Documentation.
3.4 pre-tests
- does calling 500@ekiga.net in the same time on two machines on the same NAT-ed networks work?
Ekiga 3.2.6 Pre-tests
| Feature (date/test case) | Result | Caller (software + version) | Callee (software + version) |
|---|---|---|---|
| Message Waiting Indications Support | ??? (no test case) | ||
| Contact's presence display
Away, Do Not disturb, Online status | OK, but NOK sometimes #595051 | Eugen/Yannick For all tests, Yannick used | Eugen/yannick |
| Custom message | OK | Eugen/yannick | Eugen/yannick |
| Automatic addition of network neighbors contacts | OK | Eugen | |
| Connection quality meter in the statusbar | OK (no clue if accurate) | Eugen/yannick | Eugen/yannick |
| call-in notification | OK (should work with Asterisk, not with ekiga.net) | Damien tested with Asterisk | |
| Call hold, Suspend/resume audio/video | OK | Eugen/yannick | Eugen/yannick |
| Call Transfer | OK (test case: one calls another and transfer him to 500) | Eugen/yannick | Eugen/yannick |
| Call Forwarding on
No Answer, on Busy, Always | OK (test case: setup call forwarding on always to sip:500@ekiga.net and someoine call you; he should be redirected to 500) | Eugen/yannick | Eugen/yannick |
| DTMFs Support | Seems OK (test case: upon calling 500, # terminates the call) | Eugen/yannick | Eugen/yannick |
| Calls History | OK | Eugen/yannick | Eugen/yannick |
| Call Monitoring (stats) | OK (no clue if accurate) | Eugen/yannick | Eugen/yannick |
| Enum Support | ??? No test case and: ENUM lookup doesn't work if registered to any SIP servers #589119 | ||
| Full-Screen Video | OK if hardware acceleration (NOK if using software support, and sometimes crashes #593067) | Eugen/yannick | Eugen/yannick |
| Video Resolution
176x144 320x240 640x480 704x576 | OK (not tested extensively) | Eugen/yannick | Eugen/yannick |
| SIP Capabilties exchange (codec selection) | More or less OK (not tested extensively, but tested for audio and video) Sometimes works as expected, but we found this issue #593173 | Eugen/yannick | Eugen/yannick |
| Instant Messaging with built-in smiley support | OK (Needs more extended testing...) | Eugen/yannick (Ekiga/OPAL/PTLIB from 20090909) | Eugen/yannick (Ekiga/OPAL/PTLIB from 20090909) |
| LDAP Support | OK (tested with Ekiga.net) | Eugen/yannick | Eugen/yannick |
| Configuration Assistant | Probably OK... | ||
| Configurable Sound Events | |||
| Configurable Port Ranges | |||
| STUN | |||
| Devices Auto-Detection +hotplug | OK | Yannick (works on my MSI wind using Fn key for webcam) Eugen OK too | |
| Audio codecs
iLBC, GSM-06.10, MS-GSM, G.711-Alaw, G.711-uLaw, G.726, G.721 Speex 8 Speex 16 G.722 CELT 32 CELT 48 | PCMU OK (echo test) | Yannick | |
| Echo Cancellation | |||
| Silence Detection | OK | ||
| Video codecs
H.261 H.264 H.263 H.263+ MPEG4 THEORA | H.261 OK Theora OK | Eugen/Yannick | Eugen/Yannick |
| Register to Several Accounts | OK for Eugen | Eugen/yannick | |
| Ekiga.net echo test | OK | Eugen / Yannick using Ekiga TueSep8 17:10:11 2009+0200, OPAL r23393 and PTLIB r23393 | |
| freephonie | OK with workaround only if all the video codecs are checked off (use opal >= 02/09/2009) | ||
| H323 test call (test case?) | OK. | ||
| Outbound Proxy | |||
| SIP dialog-info notifications | |||
| Interoperability with PBX like Asterisk | |||
| Local network call | |||
| network interfaces detection with dynamic addition and removal | |||
| Issue at Startup | |||
| Issue on Exit | Freeze #593064 Do not close porperly and stuck on INVITE #593074 | ||
| Crash | Crash when registering very soon after unregistering #593071 | ||
| Freeze | Freeze when Hang Up #586913 | ||
| Important but unclassified issues | TCP support for SIP #341518 Please cleanup ALSA backend #577498 |
Using VirtualBox with the "NAT" setting for Network will help testing STUN support for the next release.
Ekiga 2.9 Pre-tests
| Tester (date) | Caller (software + version) | Callee (software + version) | Presence indication | Text chat | Audio (codec) | Video (Codec + size) |
|---|---|---|---|---|---|---|
| Yannick
(16-08-2008) | Ekiga 2.9
(6617) | Ekiga 2.9
(6617) | Not tested
Test with myself: Ok | Ok | Ok for all, but
SPEEX:Broken, G726-16:Crash | Theora 320x240:Ok |
| Freechelmi
(16-08-2008) | Ekiga 2.9
(16-08-08 ) | Ekiga 2.9
(16-08-08) | 3 Status & custom variants OK on Both Sides. | NOK : Will Work till the end of the current Window lines available, new lines won't be displayed ( No automatic scrolling & refresh) | TODO | TODO |
| Yannick-Eugen
(20-08-2008) | Ekiga 2.9
20080814 (theora 640x480) | Ekiga 2.9
20080818 (theora 320x200) | ok, both way | PCMU ok | Theora 320x240 works (one way bug fixed as of 20080903)
| |
| Yannick-Eugen
(20-08-2008) | Ekiga 2.9
20080814 (h264 640x480) | Ekiga 2.9
20080818 (h264 320x200) | ok, both way | PCMU ok | h264 320x240 works (one way bug fixes as of 20080903)
| |
| Yannick-Eugen
(20-08-2008) | Ekiga 2.9
20080814 (h261 640x480) | Ekiga 2.9
20080818 (h261 320x200) | ok, both way | PCMU ok | h261 320x240 from eugen and qcif from yannick !! both way works no matter who call first quality slider works with 20080830 | |
| Yannick-Eugen
(20-08-2008) | Ekiga 2.0.12 Eugen, libpt-1.10.10
(h261) | Ekiga 2.9 Yannick
20080818 (h261 320x240) | chat ok, both way | Eugen calls me: with no audio codecs=no audio and no video!! (h261 was selected) Eugen calls me: PCMU and PCMA , GSM ok | If audio works (pcmu, pcma) video is good both way, quality slider works for 2.0.12, quality slider for 2.9=freeze and no more video transmission from 2.9 | |
| Yannick-Eugen
(31-08-2008) | Ekiga 2.9 Yannick | Ekiga 2.0.12 Eugen | 2.9 call 2.0.12: it rings on 2.0.12 side, call can be accepted (PCMU-h261), on 2.9 side there is "call redirected", no video or audio transmission. 2.9 can *not* hangup. ref. http://bugzilla.gnome.org/show_bug.cgi?id=550116 |
