System and method for reconstructing sensor locations in radiographic images

Field of the invention

The present invention generally relates to an imaging system and more specifically to a method for spatiotemporally reconstructing or estimating the locations of sensors positioned in a body organ which is imaged, for example, radiographically, and to a system that uses the sensor locations reconstruction/estimation method for localizing such sensors.

Background

A variety of medical imaging technologies is available for producing images of the interior of the human body, for example for diagnostic purpose. Radiography (an imaging technique that uses electromagnetic radiation other than visible light, for example X-rays, to view the internal structure of an object such as the human body) is frequently used for this purpose, and fluoroscopy is an imaging technique used by physicians/radiologists to obtain real-time moving image of internal organs or structures of a patient (e.g., small bowel, colon, anorectum, or other parts of the gastrointestinal (GI) system, blood vessel, etc.) through the use of a fluoroscope. Such images are typically used during surgery, for example, in order to ensure that a stent or screw is inserted correctly.

It is also known that a contrast material may be introduced into the patient to help mark anatomy parts as part of a study using fluoroscopic imaging. The contrast material may reveal functioning of, for example, blood vessels or the GI tract. Known contrast materials may include, for example, barium in the form of barium sulfate (BaSO4), which may be administered orally or rectally for GI tract evaluation, and iodine in various proprietary forms. These contrast materials absorb or scatter significant amounts of x-ray radiation and may be used with real time imaging to demonstrate dynamic bodily processes, for example esophageal peristalsis. (Esophageal peristalsis refers to the contraction of muscles in the esophagus to push forward food and liquids through the esophagus to the stomach.

There are conventional imaging systems, one of which is described in U.S. Patent Publication No. 2009/0257554, that use a contrast material to visualize and display body organs or structures, for example during operation. Sensors are often positioned in a body organ (e.g., by using a catheter) to be imaged in order to measure and display physiological parameters (e.g., peristaltic pressure) that pertain to the visualized body organ. In order to interpret the measurements correctly, which is a prerequisite to useful analysis, each measurement has to be associated with the correct sensor and with the correct sensor's location in images of the radiographically imaged organ. However, as the contrast (radiopaque) material moves in the body organ, for example in the esophagus (e.g., during a patient swallowing), it may occlude or hide one or more sensors partly, entirely, completely or to some extent, and thus render sensors radiographically invisible/indiscernible. The level of invisibility of sensors depends, among other things, on the density and propagation pattern of the contrast material in the body organ. That is, sensors may be indiscernible in some images, partly or fully discernible in other images, and very, or faintly discernible in other images.

Summary

While using contrast material may be beneficial in imaging body organs, it would be beneficial to have a system and method that reliably reconstruct sensor locations in each radiographic image even though the contrast material may, at times, radiographically occlude some of the sensors, for example while it moves in the body organ.

Embodiments of the invention, which may be directed to estimation of locations of radiopaque sensors in radiographic images, may comprise using two groups of chronologically captured (e.g., ordered by time of capture) ‘good’ radiographic images, in which many sensors are discernible, detectable, or at least partially visible (for example by not being occluded or opacified by a contrast material), to determine candidate sensor locations in a group of n chronologically captured ‘bad’ images, B 1 , B 2 , . . . , Bn, in which all or many sensors are indiscernible or not detectable in the image as sensors (for example due to the sensors being occluded or opacified by a contrast material). The images selected for the groups of good images and the group of bad images may be selected such that images in each group are mutually ordered chronologically (e.g., by using timestamps), and the group of bad images, as a whole, is chronologically interposed or placed between the two groups of ‘good’ images. A ‘good’ image may be an image that includes a number of radiographically discernible sensors that is equal to or greater than a threshold number/value. A ‘bad’ image may be an image that includes no radiographically discernible sensors at all, or a number of radiographically discernible sensors that is less or lower than a/the threshold number.

Embodiments of the invention may comprise using two iterative processes:

a first iterative process that ‘pervades’, or cascades, in one ‘direction’ (e.g. a direction in a series of images) from the first ‘bad’ image, B 1 , to the last ‘bad’ image, Bn, to determine a first set of candidate sensor locations (S.sub.Cli.sup.1) (‘i’ denotes bad image Bi) for each bad image Bi, and

a second iterative process that pervades, or cascades, in the opposite direction from the last ‘bad’ image, Bn, to the first ‘bad’ image, B 1 , to determine a second set of candidate sensor locations (S.sub.Cli.sup.2) in each bad image Bi. The first iterative process may be based on or commence using sensor locations that are estimated (e.g., reconstructed or determined) in the chronologically last image of the group of good images chronologically preceding the group of bad images. The second iterative process may be based on or commence using sensor locations that are estimated (e.g., reconstructed or determined) in the first good image following the group of bad images.

The actual location (S.sub.L.sub._.sub.k) of a particular sensor (k) in a particular bad image, Bi, may be determined based on comparison of, or from, the pertinent first and second candidate sensor locations (e.g., S.sub.CL.sub._.sub.k.sub._.sub.i.sup.1 and S.sub.CL.sub._.sub.k.sub._.sub.i.sup.2) determined for the particular sensor in the particular bad image Bi. The comparison process may be done (e.g., it may be meaningful), for example, only if each of the two sets of candidate sensor locations determined or calculated for this particular Bi contains a candidate sensor location for this specific sensor (k). For example, a sensor location (S.sub.L.sub._.sub.k) estimated for a sensor k in a bad image Bi may be a function of a first candidate sensor location and a second candidate sensor location (e.g., S.sub.CL.sub._.sub.k.sub._.sub.1.sup.1 and S.sub.CL.sub._.sub.k.sub._.sub.i.sup.2, respectively) determined for the particular sensor (e.g., S.sub.L=f(S.sub.L.sup.1, S.sub.L.sup.2)). For example, a location estimation S.sub.L of a sensor k in a bad image Bi may be determined to be either one of the two pertinent candidate sensor locations (e.g., S.sub.CL.sub._.sub.k.sub._.sub.i.sup.1 or S.sub.CL.sub._.sub.k.sub._.sub.i.sup.2), or a location that is derivable from these two candidate sensor locations. For example, location estimation S.sub.L of sensor k may be an average of the two pertinent candidate sensor locations, or a location in-between the two candidate sensor locations.

A first candidate sensor location and a second candidate sensor location determined for a particular sensor in a particular image Bi may be compared, and if the two candidate sensor locations are consistent (e.g., in agreement, overlap or coincide), within a predetermined margin, then the location of the particular sensor may be estimated from, or using, the compared, or consistent, candidate sensor locations. Two candidate sensor locations may be regarded as consistent or congruous if they have identical or similar coordinates in the image, or a distance (e.g., measured in number of pixels) between the two candidate sensor locations is less than a threshold value. If the two candidate sensor locations are not consistent/congruous, either candidate sensor location, or both candidate sensor locations, may not be eligible for estimating a sensor's location and, therefore, it/they may be discarded, or simply ignored.

An additional process comprising estimation of a location of a particular sensor in a particular image, may be performed with respect to already known location of other sensors in the same particular image, and/or with respect to already known locations of sensors in other images.

Location of sensors that are indiscernible and cannot be visually localized by themselves, may be determined by using a variety of methods. For example, locations of indiscernible sensors may be interpolated using locations of discernible sensors. Estimation of additional sensor locations may include estimation of individual sensor locations; that is, estimation of sensor locations may be performed on individual basis.

Some embodiments for reconstructing sensor locations from radiographic images may include radiographically imaging a body organ containing a plurality of (e.g., m) radiopaque sensors to provide a stream or series of images comprising a first and second groups of chronologically captured images, and a third group of chronologically captured images interposed between the first and second groups of images, each image of the first and second groups of images comprising a number of discernible sensors that is equal to or greater than a threshold number, each image of the third group of images comprising a number of discernible sensors that is less than the threshold number. These embodiments may also comprise a step of estimating sensor locations in images of the third group of images based on locations of sensors that are already estimated in an image of the first group of images and locations of sensors that are already estimated in an image of the second group of images, and also a step of estimating additional sensor locations in images of the stream/series of images based on the already determined locations of indiscernible sensors or based on the already reconstructed locations of discernible sensors, or based on both types of sensor locations.

Also disclosed herein is a system for estimating sensor locations from/in radiographic images. In some embodiments the system may include a computing device that may be configured to receive a series of chronologically captured radiographic images (e.g., ordered by time of image capture) imaging a body organ containing plurality (e.g., m) radiopaque sensors, the series of images may include a first and second groups of chronologically captured images including, each, a number of discernible sensors equal to or greater than a threshold number, and a third group of n chronologically captured images B 1 , B 2 , . . . , Bn chronologically captured between the first and second groups of images, where each image of the third group of images may include a number of discernible sensors that is less than the threshold number. According to some embodiments the system may include, or further include, a sensor locations construction (SLC) unit that may be configured to, for example: (i) reconstruct locations of discernable sensors in the first and second groups of images (ii) use reconstructed discernible sensor locations in the first and second groups of images to reconstruct additional discernable sensor locations and determine sensor locations in these groups for indiscernible sensors, and, in addition, (iii) estimate sensor locations for both discernible and indiscernible sensors in images B 1 , B 2 , . . . , Bn based on sensor locations that are discernible in a chronologically last image of the first group of images, and sensor locations that are discernible in a chronologically first image of the second group of images. The SLC unit may also be configured to improve the overall sensor location detection, or reconstruction, in the entire series of chronologically captured radiographic images. According to some embodiments the system may include, or further include, a display device to display images with discernible sensor locations and/or with reconstructed sensor locations.

Brief description of the drawings

Various exemplary embodiments are illustrated in the accompanying figures with the intent that these examples not be restrictive. It will be appreciated that for simplicity and clarity of the illustration, elements shown in the figures referenced below are not necessarily drawn to scale. Also, where considered appropriate, reference numerals may be repeated among the figures to indicate like, corresponding or analogous elements. Of the accompanying figures:

FIG. 1A is a block diagram of a radiographic imaging system for radiographically imaging a body organ according to an example embodiment;

FIG. 1B depicts an example of distribution of radiographic contrast material in a body organ along a sensing element during a swallow according to an example embodiment;

FIG. 2A schematically illustrates a stream of radiographic images acquired for a plurality of swallow sequences according to an example embodiment;

FIG. 2B schematically illustrates a swallow sequence according to an example embodiment;

FIG. 2C shows a method for reconstructing sensor locations for radiopaque sensors in radiographic images related to a single swallow according to an example embodiment;

FIG. 3A shows a method for reconstructing sensor locations for radiopaque sensors in radiographic image(s) according to another example embodiment;

FIG. 3B shows a method for estimating or enabling determination of sensor locations in a radiographic bad image based on two sets of candidate sensor locations according to an example embodiment;

FIG. 4 shows a method for reconstructing additional sensor locations in images according to an example embodiment; and

FIGS. 5A-5H depict example radiographic images overlaid with the results of different stages of sensor location estimation process applied to a sequence/series of radiographic images of a single swallow, according to embodiments of the invention.

Detailed description

The description that follows provides various details of exemplary embodiments. However, this description is not intended to limit the scope of the claims but instead to explain various principles of the invention and the manner of practicing it.

Although embodiments of the invention are not limited in this regard, discussions utilizing terms such as, for example, “processing”, “computing”, “calculating”, “determining”, “inferring”, “deducing”, “establishing”, “analyzing”, “checking”, “estimating” or the like, may refer to operation(s) and/or process(es) of a computer, a computing platform, a computing system, or other electronic computing device, that manipulate and/or transform data represented as physical (e.g., electronic) quantities within the computer's registers and/or memories into other data similarly represented as physical quantities within the computer's registers and/or memories or other information non-transitory storage medium that may store instructions to perform operations and/or processes. Unless explicitly stated, the embodiments of methods described herein are not constrained to a particular order or sequence of steps, operations or procedures. Additionally, some of the described method embodiments, steps or elements thereof can occur or be performed simultaneously or concurrently, for example at the same point in time.

“Discernible” may refer to a sensor that is visible in an image. (Such sensors are relatively easily detectable by a processor using an algorithm.) “Indiscernible” may refer to a sensor that is invisible in an image. (Such sensors are usually undetectable by a processor, hence the use of sensor locations estimation methods disclosed herein.) “Reconstruction” of a sensor location may refer to, or include, a process by which a discernible sensor is detected in radiographic images, and its location in the image is defined using a coordinate system and, optionally, presented visually. “Determination” of a sensor location may refer to, or include, a process by which a location of an indiscernible sensor in an image is ‘guessed’ or inferred using other sensor locations (e.g., locations of discernible sensors and/or sensor locations determined for other indiscernible sensors). “Determination” is also used in the context of determination of a candidate sensor location. “Estimation” of a sensor location may refer to, or include, a process by which a location of a discernible sensor in an image is obtained through detection of the sensor in the image, and also a process by which a location of an indiscernible sensor in an image is ‘guessed’ using two candidate sensor locations.

The following notations are used herein: (i) S.sub.CLi.sup.1 and S.sub.CLi.sup.2, respectively, are or represent a first set of candidate sensor locations and a second set of candidate sensor locations that are estimated for sensor locations in a given bad image Bi (i=1, 2, . . . , n), (ii) S.sub.CL.sub._.sub.k.sub._.sub.i.sup.1 is or represents a first candidate sensor location for sensor k in image Bi (S.sub.CL.sub._.sub.k.sub._.sub.i.sup.1 is an ‘element’ of set S.sub.CLi.sup.1), (iii) similarly, S.sub.CL.sub._.sub.k.sub._.sub.i.sup.2 is or represents a second candidate sensor location for sensor k in image Bi (S.sub.CL.sub._.sub.k.sub._.sub.i.sup.2 is an ‘element’ of set S.sub.CLi.sup.2), and (iii) S.sub.L is an estimation of a ‘true’ (genuine) location of a sensor in a radiographic image. When discussed herein a “location” of a sensor may be the location of an image of the sensor within an image (e.g. a radiographic image, a photographic image, an ultrasound image, or other image). Similarly, when discussed herein, a sensor may be an actual real-world sensor, or an image of a sensor within an image. Thus an “image comprising a sensor” may refer to an image including an image of a sensor.

Displaying a location of a (radiopaque) sensor in a radiographic image may mean or include estimating the location of the sensor in the image and altering the (original, or raw) image to include in it (e.g., by addition of) a graphical object (e.g., “+”, “x”, “o”, or another shape, notation or symbol) whose positional location (e.g., coordinates) represents or indicates the location of the sensor in the image. Displaying a sensor's location may mean or include, for example, visually superimposing an image of, or that includes, a graphical object on the radiographic image in a way that the graphical object, while superimposed on the radiographic image, represents or indicates the location of the sensor in the radiographic image.

FIG. 1A is a block diagram of a visualization system 100 according to an example embodiment of the invention. Visualization system 100 may include an imaging system 120 for obtaining a series of successive radioscopic images of a region of interest (ROI) 110 in or of a patient's body, a sensors reading unit (“SRU”) 122 for reading or measuring, or otherwise detecting sensor output values, a computing system or device 130 for receiving image data (and possibly other data) from imaging system 120 and sensors' information (e.g., identification information, output values) from SRU 122 , and for processing the image data and the sensors information, and a display device 180 (e.g. a monitor) for displaying any of radioscopic images, estimated sensor locations and, optionally, sensors' output values (e.g., pressure values).

Before the radioscopic imaging/study/procedure is commenced, a sensing element 112 (e.g., a catheter with discrete sensors disposed thereon or therein) may be inserted into, and positioned or placed in, the body organ whose physiological functionality is to be studied or examined. Sensing element 112 may include m sensors, designated as S 1 , S 2 , . . . , Sm. Sensors S 1 , S 2 , . . . , Sm may be inserted into a body organ or portion in the ROI by using, for example, a catheter, or by using any other suitable method. When imaging system 120 images ROI 110 , it also images all or some of the sensors of sensing element 112 . For example, the number of sensors that are ‘successfully’ imaged (that is, clearly visible or discernible) in each image may vary from one image to another depending on for example, whether, or on the extent to which, the sensors are radiographically occluded by a contrast material that is administered into the body organ or portion and moves or flows, for example, by applied peristalsis, in ROI 110 . (The contrast material may absorb or scatter significant amounts of x-ray radiation, and thus may be conspicuous radiographically.) Other reasons for sensors not being visible or discernable may occur. The body organ, portion or lumen may be or include, for example, the esophagus, a blood vessel, etc., in which m radiopaque sensors S 1 , S 2 , . . . , Sm are disposed, distributed or placed.

Embodiments of the invention may include imaging the body organ or lumen and the m radiopaque sensors over time, for example such that a series of images are successively acquired one image at a time during a time period including one or more swallow procedures. Such imaging may be performed, for example, in order to provide a chronologically captured stream, or series, of radiopaque images (e.g., by outputting a signal or data representing these images) corresponding to one or more swallow procedures. With each swallow procedure may be associated a swallow sequence of images captured before, during and after a swallow of radiopaque (contrast) material. Embodiments of the methods described herein may be applied, for example, to each swallow sequence of images, as described herein.

The physical/actual spacing between sensors S 1 , S 2 . . . , Sm, as well as the shape and dimensions of each sensor, may be known a priori and used to identify the sensors in radioscopic images. One or more sensors may be designed slightly differently than others in order for them to serve as fiducial indicia, or reference markers, in radioscopic images, in order to facilitate identification of sensors in radioscopic images. Special (e.g., dedicated) radio discernible markers may be built into the sensing device, or a radiographic ‘signature’ of certain non-sensing components of the sensing device may be used, to facilitate identification of the ordered sensors from images. Sensors S 1 , S 2 . . . , Sm may be or include, for example, pressure sensors, pH sensors, temperature sensors, etc., or a combination of sensors of different types (e.g., pressure sensors and pH sensors).

Commercially available medical imaging system may be used to image a region of interest like or similar to ROI 110 . Radio discernible sensors 112 may have any suitable size, shape, and material composition that may render them detectable by computing device 130 from radiographic images that are produced by imaging system 120 . Imaging system 120 may image the ROI prior to the introduction of a contrast material to establish a baseline image. Once a suitable contrast material is introduced into ROI 110 , imaging system 120 may acquire a series of images.

Computing device 130 may be of a type known in the art for processing medical image data, and it may interoperate with imaging system 120 to receive the series of successive radioscopic images for processing. Computing device 130 may be configured to perform steps of the methods disclosed herein, such as the image processing and display methods disclosed herein, by using computer-executable modules or instruction codes that may be stored, for example, in a storage unit 172 , and executed by a suitable controller/processor (e.g., controller/processor 170 ). For example, processing units such as controller/processor 170 , image processor 140 , sensor locations construction (“SLC”) unit 150 , sensors reading unit (“SRU”) 122 , etc. may be configured to perform methods disclosed herein by, for example, executing code or software and/or including dedicated circuitry. Processors and units such as 170 , 140 , 150 , 122 etc., may be or may be part of a computing system. SLC unit 150 , or only functions thereof, may be embedded in or performed by controller/processor 170 . Computing device 130 may also include image processor 140 and SLC unit 150 . Image processor 140 may process radioscopic image data that it receives from imaging system 120 , for example, to identify the body lumen or organ. SLC unit 150 , possibly in conjunction with processor 140 , may process the radioscopic image data, for example, to detect and identify sensors, or sensor locations S 1 , S 2 , . . . , Sm (or some of them) in each radioscopic image, and, if required, to estimate, and optionally display, locations for sensors (indiscernible or discernible, or sensors of both types) in one or more images. Sensor Outputs Reading, and Association of Output Values with Imaged Sensors

Sensors reading unit (SRU) 122 , which may be integrated into computing device 130 , for example as interface module, may be configured to read or measure, or otherwise detect, the output values of sensors S 1 , S 2 , . . . , Sm serially or in parallel. Since detection of sensor outputs is faster than the imaging process, the output of the sensors may be detected, for any image, in series, one sensor output after another. SRU 122 may transfer the sensor output readings (values) to computing device 130 , and computing device 130 , by using, for example, processor 170 , may respectively associate the sensors' output values with the sensor locations estimated by SLC unit 150 , in the related image. If SLC unit 150 cannot, or does not, identify a sensor in an image, it may infer its location in the image from identified locations of other sensors and/or from sensors identified in previous or subsequent images when multiple images are processed, such as in a video stream, and computing device 130 may handle ‘inferred’ sensor locations as if they were identified; i.e., in the same way as it handles identified sensors.

By using the information transferred from SRU 122 (e.g., sensors' identification codes, sensors' output values, etc.), computing device 130 may ‘know’ which sensor that computing device 130 identifies in the image is the first sensor (e.g., topmost in the image, or leftmost in in the image, etc.), the second sensor, and so on, and which measured output value is associated with (read/measured from) which sensor. SRU 122 may be a separate apparatus or a component thereof or embedded in computing device 130 . Estimating Locations of Indiscernible Sensors

As explained herein, one or more sensors of sensing element 112 , which may be positioned in the body organ that is imaged or that is to be imaged, may not be identifiable or detectable in a radioscopic image, for example due to it/them being radiographically indiscernible because it/they is/are ‘hidden’ (e.g., occluded, opacified or overlapped) by the radiographic contrast material, or because of a different reason. If SLC unit 150 cannot identify a particular sensor because it is occluded or opacified (e.g., sensor S 7 is at 114 , though indiscernible), it may estimate, deduct or infer the location of the sensor (e.g., sensor S 7 ) using any of the methods disclosed herein.

Based on a priori information related to sensing element 112 , such as the number and size of the sensors and the spacing between them, SLC unit 150 may reconstruct discernible sensor locations and estimate, determine or calculate, the ‘expected’ location(s) (e.g., coordinates) of the unidentified/indiscernible sensors or sensor locations.

SRU 122 and SLC unit 150 may be implemented in software or in hardware, or partly in hardware and partly in software: they may be dedicated hardware unit, or they may be a code or instructions executed by a controller/processor, e.g., image processor 140 or processor 170 . The code/instructions may be distributed among two or more processors. For example, processor 170 , by executing suitable software, code or instructions, may carry out steps which are performed by SRU 122 and SLC unit 150 , and other functions in computing device 130 , and thus may function as these units. Every process or method step described herein, or some of them, may be performed solely by sensor locations construction (SLC) unit (e.g., module, circuit) 150 or processor 170 , or jointly by SLC 150 and processor 170 . In places where it is stated that a controller or processor performs a process and/or method steps, the process and/or method steps may be performed by SLC unit 150 , which may be a dedicated electrical module or circuit.

FIG. 1B illustrates an example distribution of radiographic contrast material 192 along a sensing element 190 during swallow of a contrast material according to an example embodiment of the invention. A sensing element 190 may include m sensors (e.g., m>12), only nine of which are shown in FIG. 1B as an example, that is, sensors S 4 through S 12 . Discernibility of a sensor in a radiographic image depends, among other things, on characteristics of the sensor and on the density of the contrast material interposed or placed between the sensor and the radiographic energy source.

A sensor may be ‘very’, or fully, discernible if the density of the contrast material is non-existent or negligible. The sensor may be discernible to some extent if the contrast material is dense to some extent, and indiscernible if the contrast material is very dense. By way of example, radiographic contrast 192 is shown having two, visually discernible, density regions, one of which is density region 194 and the other is density region 196 . Contrast material 192 is denser in density region 196 than in density region 194 , therefore density region 196 is darker (‘more’ black) than density region 194 .

Sensors S 4 , S 5 , S 11 and S 12 are very discernible because they are not occluded at all by contrast material 192 . Sensors S 6 and S 7 are partly discernible (e.g., less discernible than; e.g., sensors S 4 and S 5 ) because they are occluded by semitransparent region 194 of contrast material 192 . Sensors S 9 and S 10 are indiscernible at all because they are completely occluded/opacified by completely opaque region 196 (of contrast material 192 ), which more dense than region 194 of contrast material 192 . Part of sensor S 8 is discernible in a similar way as sensors S 6 and S 7 , and the other part of sensor S 8 is indiscernible at all, as are sensors S 9 and S 10 . (Sensors partly occluded by contrast material region 194 are made visually conspicuous, or accentuated, using a white, broken-lined, rectangle 198 . Similarly, since sensors S 9 -S 10 are completely indiscernible, their expected or ‘guessed’ locations are shown using white, broken-lined, rectangles.)

FIG. 2A schematically illustrates/shows a series 200 of chronologically captured radioscopic images (e.g., ordered by time of capture), with radioscopic image 202 being one image. The description related to FIG. 2A and to other drawings mentions the esophagus, so the contrast material is administered by swallowing. However, the esophagus is only an example body organ. A catheter with radiopaque sensors may be positioned in other body parts/lumens, for example in a blood vessel, and other contrast material administration methods may be used, which are suited for the imaged body organ/lumen.

Image series 210 , which may refer to an example swallow procedure, comprises a first group or subseries of images (group 220 ) captured before the contrast material is administered to the body organ, a second group or subseries of images (group 230 ) captured while the contrast material moves in the body organ, and a third group or subseries of images (group 240 ) captured after the contrast material is diffused or cleared from the body organ.

Series 200 of radioscopic images may include images related to one swallow of contrast material or to multiple such swallows. Series 200 may be arranged chronologically by time of capture. The methods disclosed herein are applicable to individual swallows (e.g., individual times a patient swallows). For example, chronologically captured images related to one swallow procedure are shown at 210 . Monitoring an individual swallow (e.g., by a system similar to system 100 of FIG. 1A ) may result in or include obtaining a pre-swallow (contrast material free) series of chronologically captured radiographic images 220 , then obtaining a second series of chronologically captured images 230 during which a contrast material is administered (e.g., swallowed) and moves in or through the body organ (and the series of radiographic images are obtained while the contrast material is moving (e.g., in the esophagus)), then (after emptying of, or clearing, the contrast material from the body organ) obtaining a third contrast material free series of chronologically captured radiographic images 240 . (A sensing element identical or similar to, for example, sensing elements 112 of FIG. 1A is disposed or positioned in the body organ (e.g., esophagus) when images 210 are taken one image at a time.)

Image series 220 and 240 may respectively provide two groups of chronologically ordered ‘good’ radiographic images 250 and 270 . An image regarded or classified as a ‘good’ radiographic image is a radiographic image in which all, most or many of the sensors are radiographically discernible; e.g., their locations clearly appear or detectable in the radiographic images. (Image series 250 may be, for example, a subseries of image series 220 , and image series 270 may be, for example, a subseries of image series 240 .)

Image series 230 may provide a group of chronologically ordered ‘bad’ radiographic images 260 . (Image group 260 may be, for example, a subseries of image series 230 .) An image regarded or classified as a ‘bad’ radiographic image is a radiographic image in which all, most or many of the sensors are indiscernible, for example, at all, or they are discernible poorly such that their location cannot be determined at all or reliably. (Group 260 of bad images may be chronologically interposed between group 250 of good images and group 270 of good images.)

Image series 220 and 240 may enable, for example, a controller/processor, such as processor 170 of FIG. 1A to detect, identify or single out an individual swallow (swallow procedure) and, in addition, the processor may use image groups 250 and 270 of good images, which it may respectively extract or select from image series 220 and 240 , to determine sensor locations in bad images 260 . ‘Determining a sensor location’ generally refers to, or uses, a process by which a location of an indiscernible sensor, for example in a bad image, is estimated, and the image subject of the sensor location estimation process is modified or altered such that it visualizes also the sensor's estimated location (e.g., by superimposing a graphical object that indicates the sensor's estimated location on the image).

FIG. 2B schematically illustrates/shows groups of good and bad images of a single swallow according to an example embodiment of the present invention. After sub-series 210 of images, which is related to an individual swallow, is identified within series 200 of radiographic images, images 210 undergoes the sensor location estimation methods disclosed herein. Initially, a quality criterion may be used in order to classify each image in sub-series 210 as a good image or as a bad image. As described herein, while images taken/captured during monitoring phases 220 and 240 are characterized by providing more good images than bad images, images taken/captured during monitoring phase 230 are characterized by providing more bad images than good images.

Good images and bad images may be grouped chronologically; namely, according to the order in which the images were taken, captured, generated or provided by/from a system similar to system 100 of FIG. 1 . Referring to FIG. 2A , good images obtained during monitoring phase 220 may be grouped into a first group 250 of good images; bad images obtained during monitoring phase 230 may be grouped into a group 260 of bad images, and good images obtained during monitoring phase 240 may be grouped into a second group 270 of good images.

Each bad image, B, of group 260 of a number n of bad images may be indexed (Bi, i=1, 2, . . . , n) according to the chronological order in which the images were taken, captured, generated or obtained. For example, a first bad image in group 260 may be indexed B 1 , as shown in FIG. 2B ; the next (chronologically contiguous) bad image may be indexed B 2 , and so on, where the last bad image in group 260 may be indexed Bn, as shown in FIG. 2B . Indexing of the bad images may be performed in the order described herein, for example as described above. However, the bad images may be indexed in reverse order.

Reference numerals 280 and 290 respectively denote a chronologically last good image of group 250 of good images and a chronologically first good image of group 270 of good images. Good images 280 and 290 are chronologically contiguous to group 260 of bad images, though to opposite sides/ends of group 260 . (Last good image 280 of group 250 of good images may be referred to as an ‘edge good image’ of group 250 ; first good image 290 of group 270 of good images may be referred to as an ‘edge good image’ of group 270 .) For example, edge good image 280 is chronologically contiguous or adjacent to a first, chronologically captured, image of group 260 of bad images (for example to first bad image B 1 ), and edge good image 290 is chronologically contiguous or adjacent to a last, chronologically captured, image of group 260 of bad images (for example to last bad image Bn). Bad image B 1 , the first bad image of bad image group 260 , is chronologically captured or obtained after image 280 . Bad image Bn, the last bad image of bad image group 260 , is chronologically captured or obtained after good image 280 and before good image 290 .

The description continues in the full USPTO document.