DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments/Amendments
Applicant's amendments filed February 13, 2026 have been fully considered.
Objections to the Drawings and Claim 18 are withdrawn in response to applicant’s correction made in the amendments.
Applicant's arguments filed on February 13, 2026 have been fully considered but they are not persuasive. The 35 USC § 103 rejections of independent claims 1, 10, and 19 are maintained.
Applicant argues the combination of Bulzacki, Vaidya, and Schulte, taken alone or in combination, fail to disclose "determining... scene histograms from the normalized color information of the first 3D image, each of the scene histograms determined for a voxel of the plurality of voxels of the first 3D image," as recited in independent claims 1, 10, and 19. Applicant also argues that the combination relies on a hindsight reconstruction. However, the examiner respectfully disagrees.
Bulzacki discloses rendering 3D models of objects. A voxel is a pixel in a 3D image. Therefore, Bulzacki’s color histogram is based on pixels from a 3D image. Vaidya is only relied upon on explicit disclosure of identifying “voxels” from a 3D image, which is a pixel in 3D. Further, Vaidya also teaches voxel-specific histogram as described in Vaidya paragraph 89.
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, and Vaidya to determine, by the processor, a plurality of voxels of the 3D image, each voxel including one or more of the 3D points of the plurality of 3D points; and to determine scene histograms from the normalized color information of the first 3D image, each of the scene color histograms determined for a voxel in the first 3D image.
The reason for doing so is to accurately measure the length, width and height of a typically cuboidal object in the field of view in the presence of noise (Vaidya: ¶7).
Applicant further argues that Schulte’s histograms are based on 2D pixel grids and not 3D volumetric voxels. However, Schulte is only relied upon to teach determining a presence of a surface of an object from a color score.
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Vaidya, and Schulte to include: and determine the presence of a surface of an object in the first 3D image from the color score.
The reason for doing so is to increase the speed and accuracy of search results (Schulte: ¶4).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-8, 10-12, and 15-19 are rejected under 35 U.S.C. 103 as being unpatentable over Bulzacki et al. US (20250131788 A1, hereinafter “Bulzacki”) in view of Vaidya et al. (US 20220358668 A1, hereinafter “Vaidya”) and Schulte (US 20180082410 A1, hereinafter “Schulte”).
Regarding claim 10:
Bulzacki teaches:
A system (Bulzacki: ¶5, "Embodiments described herein provide a platform, device and process for monitoring game activities at a gaming table. In particular, embodiments described herein provide a platform, device and process for capturing images of the surface of a gaming table and determining the quantity, identity, and arrangement of chips bet at a gaming table") for performing surface matching, the system comprising:
a 3D imager configured to capture and provide 3D images of a field of view of the 3D imager (Bulzacki: ¶229, "FIGS. 5 to 7 illustrate example images taken from a bet recognition device mounted on a gaming table according to some embodiments. The image data for the images may include depth data taken from a table-mounted depth camera. The example images illustrate the table and bet area rendered in three-dimensions (3D). The image data defines stacks of chips in the bet areas in 3D defined using X, Y, and Z coordinate values. The game monitoring server 20 may represent a gaming table surface as a model of X, Y and Z coordinate values including betting areas"),
the 3D images including (i) a plurality of 3D points of the field of view (Bulzacki: ¶121, "Embodiments described herein can be used to capture images of the surface of a gaming table, including gambling chips, in response to certain events such as placing a bet. A device included in some embodiments is configured to remove background image data, for example, by distinguishing chips from the background based on their respective depth values. The device is configured to identify points of interest on the chip images, including across the length of a side view of each chip, and classify each point of interest using histogram data of image channels corresponding to the point of interest. . ."; Bulzacki: ¶461, "In some embodiments, image processing engine 204 is configured to calibrate image data of the same gaming objects to generate depth data. In some embodiments, image processing engine 204 uses other image data, for example, IR data, in combination to generate depth data, decide how far away an object is, or decide on an object's location. In some embodiments of the example embodiment, image processing engine 204 determines a distance of an object based on how light or a light pattern warps over an object as captured by an IR imaging component. This distance information is compared with other distance information of the same object captured by a separate imaging component located a known distance away. In some embodiments of the example embodiment, two RGB cameras are used to generate depth information using triangulation"; note: the depth information being the plurality of 3d points)
and (ii) color information of the field of view (Bulzacki: ¶186, "Image data, along with other metadata may be encapsulated in the form of information channels that may be use for transmission and/or otherwise encoded. In some embodiments, 10 or more channels of information are provided by the bet recognition device 30, and the channels may include, for example, image data taken with different color balances and parameters, image data from different sensors, metadata, etc."; note: also see ¶ 42-43, 242, 375);
a processor and computer-readable media storage having machine readable instructions stored thereon that, when the machine readable instructions are executed, cause the system to (Bulzacki: ¶477, "Throughout the foregoing discussion, numerous references are made regarding servers, services, interfaces, portals, platforms, or other systems formed from computing devices. It should be appreciated that the use of such terms is deemed to represent one or more computing devices having at least one processor configured to execute software instructions stored on a computer readable tangible, non-transitory medium. For example, a server can include one or more computers operating as a web server, database server, or other type of computer server in a manner to fulfill described roles, responsibilities, or functions"):
obtain a first 3D image of the field of view of the 3D imager (Bulzacki: ¶229, "FIGS. 5 to 7 illustrate example images taken from a bet recognition device mounted on a gaming table according to some embodiments. The image data for the images may include depth data taken from a table-mounted depth camera. The example images illustrate the table and bet area rendered in three-dimensions (3D). The image data defines stacks of chips in the bet areas in 3D defined using X, Y, and Z coordinate values. The game monitoring server 20 may represent a gaming table surface as a model of X, Y and Z coordinate values including betting areas");
normalize color information of the first 3D image into a common color space (Bulzacki: ¶375, "At 5733, processor 204 is configured to generate rotation and scale invariant image data, for example, for each region of interest or bet volume. This allows the size or scale of the regions of interest (e.g., chip stack) or portions thereof to be represented in accurate scale when resized or rotated. Processor 204 also can be configured to normalize the image data for each region of interest. At 5734, processor 204 can be configured to extend channels by transforming the image, for example, transforming the image from RGB to a different color space where intensity is decoupled from color information or luminance is decoupled from chrominance. Examples of such color spaces include L*a*b*, HSV, and HSL. For example, processor 204 can be configured to generate YCbCr image data based on image data captured from red, green, and blue channels. The YCbCr colour space can be used because in the YCbCr colour space, luminance is decoupled from chrominance, i.e., intensity is decoupled from color. This can improve the subsequent processing, transmission, and storage features and quality of the image data. For example, the luminance and chrominance can be processed separately. This can allow for improved compression of the data (e.g., for reduced bandwidth consumption during transmission) while enabling a satisfactory representation of the image from the compressed data");
determine a color score from at least the scene histograms and model histograms, the model histograms being indicative of color information of a model image of a 3D object (Bulzacki: ¶253, "Waveforms may be extracted in relation to each chip (e.g., as extracted from the available pixels in image data for each channel of information (note: RGB channels as described in ¶142), and these waveforms may represent, for example, information extracted from histogram information, or from other image information. For example, a Fourier transform may be conducted on the image data separately from extracted histogram information. In some embodiments, a histogram and a Fourier transform are used in combination"; ¶254, "A best-matching waveform approach may be utilized to estimate which color and/or markings are associated with a particular chip. For example, each chip may have a corresponding waveform (for each image channel) and these may be used, in some embodiments, in aggregate, to classify the chips based on chip values. In some embodiments, where the data is not sufficiently distinguishing between different chip values (e.g., poor lighting makes it difficult to distinguish between pink and red), the system may be adapted to provide a confidence score associated with how closely matched the waveform is with a reference template. This confidence score, for example, may be used to modify sensory characteristics, lighting conditions, etc., so that the confidence score may be improved on future image processing. In some embodiments, the interfaces provided to users may also utilize the confidence score in identifying how strong of a match was determined between chip images and reference templates. The received signals 1602 and 1604 may be different for each type of chip, and the waveforms may be processed through a classifier to determine a “best match”. As described in some embodiments, the confidence in determining a best match may be based on (1) how well matched the chip is to a reference waveform, and (2) how much the chip is able to distinguish between two different reference waveforms. The confidence score may be used to activate triggers to improve the confidence score, for example, by automatically activating reference illumination or requesting additional images (e.g., moving the camera to get more pixels due to an obstruction, lengthening a shutter speed to remove effects of motion, temporarily allocating additional processing power to remove noise artifacts)"; ¶217, "The game monitoring server 20 may be configured for performing various steps of calibration, and the calibration may include developing an array of reference templates in relation to the particular set up of a gaming surface or gaming table. For example, the reference templates may include what chips “look like” at a particular table in view of usual gameplay settings, etc. Further, the reference templates may track lighting conditions across the span of a day, a season, in view of upcoming events, nearby light sources (e.g., slot machines), etc. New templates may be provided, for example, when new chips or variations of chip types are being introduced into circulation at a particular gaming facility. In some embodiments, such introduction of new chips may require a machine-learning training phase to be conducted to build a reference library")
NOTE 10A: The model image of a 3D object being the reference template of Bulzacki, since the reference template (model image) is compared via waveforms with the first 3d image, therefore the reference template (model image) must have its own model histogram that is being indicative of color information of a model image of a 3D object. The confidence score being the color score associated with waveforms from information extracted from histogram information as described in Bulzacki: ¶253.);
Bulzacki also teaches:
to determine, by the processor, a plurality ofof the 3d image (Bulzacki: ¶167, " . . .the image processing engine 204 may be configured to visually identify the pixels and/or regions of interest. . ."),
and determine scene histograms from the normalized color information of the first 3D image (note: normalization happens at step 5733 in reference to Bulzacki: ¶375, and histogram generation happens at step 5746 in reference to Bulzacki: ¶383),
each of the scene color histograms determined for a in the first 3D image (Bulzacki: ¶389, "A histogram can be generated for portions of image data, for example, where each portion corresponds to a single point of interest. That is, each point of interest can be represented as gradient values encoding the magnitude and direction of each pixel in the selected point of interest").
determine from the color score (confidence score associated with waveforms extracted from histogram information) as referenced in Bulzacki ¶254.
Bulzacki also teaches to determine the presence of a surface of an object in the first 3D image (Bulzacki Fig. 7 and ¶231, "FIG. 7 is a compressed representation 700, wherein non-chip imagery has been filtered out, and the image primarily consists of images of stacks of chips 702, 704, and 706. Filtering techniques, for example, include the use of edge detection algorithms (e.g., difference of Gaussians). The representation 700 may be compressed such that chips are detected among other features, so that various regions of interest can be identified. In some embodiments, representation 700 is combined with depth data so that background and foreground chips may be distinguished from one another. For example, chips that may be within a betting area indicative of a bet may also have chips that are in the background (e.g., chips that the player has in the player's stacks that are not being used for betting). Depth data may be used to distinguish between those chips that are in the betting area as opposed to those chips that are out of the betting area")
However, Bulzacki does not teach the analogous art Vaidya teaches:
Vaidya teaches:
determine, by the processor, a plurality of voxels of the 3D image, each voxel including one or more of the 3D points of the plurality of 3D points (Vaidya: ¶8, "In an illustrative embodiment, a system and method for estimating dimensions of an approximately cuboidal object from a 3D image of the object, acquired by an image sensor of the vision system processor, is provided. An identification module, associated with the vision system processor, automatically identifies a 3D region in the 3D image that contains the cuboidal object. A selection module, associated with the vision system processor, automatically selects 3D image data from the 3D image that corresponds to approximate faces or boundaries of the cuboidal object. An analysis module statistically analyzes, and generates statistics for, the selected 3D image data that correspond to approximate cuboidal object faces or boundaries. A refinement module, responsive to the analysis module, then chooses statistics that correspond to improved cuboidal dimensions from among cuboidal object height, width, and length, width and height, the improved cuboidal dimensions being provided as dimensions for the object. Illustratively, the identification module identifies the 3D region using a 3D connected component analysis and/or the selection module selects the 3D region by testing the 3D image data using the 3D connected component analysis. The 3D connected component analysis can be constructed and arranged to identify groups of voxels of the 3D image that are adjacent to each other and that excludes, from each one of the groups, any voxels whose distance from a respective of the groups exceeds an adjacency threshold. . .");
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki and implement Vaidya's teaching to determine, by the processor, a plurality of voxels of the 3D image, each voxel including one or more of the 3D points of the plurality of 3D points; and to determine scene histograms from the normalized color information of the first 3D image, each of the scene color histograms determined for a voxel in the first 3D image to accurately measure the length, width and height of a typically cuboidal object in the field of view in the presence of noise (Vaidya: ¶7).
However, the combination of Bulzacki and Vaidya still does not teach the analogous art Schulte teaches.
Schulte teaches:
determine the presence of a surface of an object in the first 3D image from the color score (Schulte: ¶65, "In one embodiment, the similarity score is used for foreground detection 712, which may include background subtraction to detect a foreground object and isolate portions of the image for further processing, such as motion tracking or image classification. For example, a threshold can be used to determine whether a block of the received image is background or a foreground object based on the similarity score. If the similarity score is above the threshold, then the block is classified as background. If the similarity score is at or below the threshold, then the block is classified as foreground. The foreground blocks represent a potential object in the field of view and may be used for further image processing, such as object classification or motion detection, or to trigger actions by the video surveillance system. In various embodiments, a video surveillance system may use object and motion detection to trigger the video surveillance system to record the image stream from the image capture device, to classify the objects and activity within the field of view, or perform other analyses and actions in accordance with the system configuration"; Schulte: ¶66, "FIGS. 8a and 8b illustrate an object classification system which may be used with the histogram comparison methods described herein. In one embodiment, an object classification system includes a learning model (e.g., support vector machine) that analyzes a series of training images 810 using oriented gradient histograms (a) for each block of the training image (b). The learning model extracts the features vector for images containing the desired object (c) (e.g., a human, animal, vehicle or other object), and may also extract the features vector for negative images that do not contain the desired object (d). The images may be classified by the histogram distribution of each block and the arrangement of blocks into the desired object (e.g., a human image). The stored images and histograms may then be used for image classification as described herein").
NOTE 10B: Schulte teaches object classification based on similarity score (color score) associated with a threshold. Objects classified whether a human, animal, vehicle, or other objects have surfaces (foreground objects), therefore, the object classification includes determining the presence of a surface of an object. The similarity score being the color score since the similarity score is calculated between the histograms for each block of the input image (Schulte: ¶64), wherein with each histogram representing the distribution of each color component in the image color space (Schulte: ¶36).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Vaidya, and Schulte to include: and determine the presence of a surface of an object in the first 3D image from the color score to increase the speed and accuracy of search results (Schulte: ¶4).
Regarding claim 11, depending on claim 10,
The combination of Bulzacki, Vaidya, and Schulte teaches:
The system of claim 10, wherein to normalize the color information into a common color space, the machine readable instructions further cause the system to convert the color information of the first 3D image into lab color space or hue, saturation, lightness (HSL) color space (Bulzacki: ¶375, "At 5733, processor 204 is configured to generate rotation and scale invariant image data, for example, for each region of interest or bet volume. This allows the size or scale of the regions of interest (e.g., chip stack) or portions thereof to be represented in accurate scale when resized or rotated. Processor 204 also can be configured to normalize the image data for each region of interest. At 5734, processor 204 can be configured to extend channels by transforming the image, for example, transforming the image from RGB to a different color space where intensity is decoupled from color information or luminance is decoupled from chrominance. Examples of such color spaces include L*a*b*, HSV, and HSL. . .").
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Vaidya, and Schulte to include: wherein to normalize the color information into a common color space, the machine readable instructions further cause the system to convert the color information of the first 3D image into lab color space or hue, saturation, lightness (HSL) color space to increase the speed and accuracy of search results (Schulte: ¶4).
Regarding claim 12, depending on claim 10,
The combination of Bulzacki, Vaidya, and Schulte teaches:
The system of claim 10, wherein to determine the scene histograms, the machine readable instructions further cause the system to determine a color histogram for each voxel of the first 3D image (see claim 10 rejection regarding determination of the scene histograms for each voxel of the first 3d image).
NOTE 12A: The scene histograms of Bulzacki is a color histogram (Bulzacki: ¶253, "Waveforms may be extracted in relation to each chip (e.g., as extracted from the available pixels in image data for each channel of information (note: RGB channels as described in ¶142)), and these waveforms may represent, for example, information extracted from histogram information, or from other image information. For example, a Fourier transform may be conducted on the image data separately from extracted histogram information. In some embodiments, a histogram and a Fourier transform are used in combination"; ¶254, "A best-matching waveform approach may be utilized to estimate which color and/or markings are associated with a particular chip). Since the waveforms are extracted from histograms and used to estimate colors associated with a chip, therefore, the histogram is a color histogram.
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Vaidya, and Schulte to include: wherein to determine the scene histograms, the machine readable instructions further cause the system to determine a color histogram for each voxel of the first 3D image to increase the speed and accuracy of search results (Schulte: ¶4).
Regarding claim 15, depending on claim 10,
The combination of Bulzacki, Vaidya, and Schulte teaches:
The system of claim 10, wherein to determine the presence of the object, the machine readable instructions further cause the system to determine the presence of the object by comparing the color score to a threshold score (Schulte: ¶65, "In one embodiment, the similarity score is used for foreground detection 712, which may include background subtraction to detect a foreground object and isolate portions of the image for further processing, such as motion tracking or image classification. For example, a threshold can be used to determine whether a block of the received image is background or a foreground object based on the similarity score. If the similarity score is above the threshold, then the block is classified as background. If the similarity score is at or below the threshold, then the block is classified as foreground. The foreground blocks represent a potential object in the field of view and may be used for further image processing, such as object classification or motion detection, or to trigger actions by the video surveillance system. In various embodiments, a video surveillance system may use object and motion detection to trigger the video surveillance system to record the image stream from the image capture device, to classify the objects and activity within the field of view, or perform other analyses and actions in accordance with the system configuration"; Schulte: ¶66, "FIGS. 8a and 8b illustrate an object classification system which may be used with the histogram comparison methods described herein. In one embodiment, an object classification system includes a learning model (e.g., support vector machine) that analyzes a series of training images 810 using oriented gradient histograms (a) for each block of the training image (b). The learning model extracts the features vector for images containing the desired object (c) (e.g., a human, animal, vehicle or other object), and may also extract the features vector for negative images that do not contain the desired object (d). The images may be classified by the histogram distribution of each block and the arrangement of blocks into the desired object (e.g., a human image). The stored images and histograms may then be used for image classification as described herein"; also see NOTE 10B).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Vaidya, and Schulte to include: wherein to determine the presence of the object, the machine readable instructions further cause the system to determine the presence of the object by comparing the color score to a threshold score to increase the speed and accuracy of search results (Schulte: ¶4).
Regarding claim 16, depending on claim 10,
The combination of Bulzacki, Vaidya, and Schulte teaches:
to obtain a second image of a model object and generating a second histogram for the second image (Schulte: ¶9, "An exemplary method for comparing two images includes receiving a source image, receiving a comparison image, generating a first histogram for the source image and generating a second histogram for the comparison image. In one embodiment, the source image is received from a network device and the comparison image is one of a plurality of stored images (e.g., a database of images). The second histogram may be generated at the time of the comparison or generated ahead of time and stored in a database for retrieval with the comparison image. The first and second histograms may correspond to any image characteristic, including a color histogram corresponding to the distribution of the intensity of a corresponding color among image pixels in the source image")
Bulzacki teaches obtaining 3D images.
In reference to claim 10 rejection, the combination of Bulzacki, Vaidya, and Schulte teaches that the first 3D image obtained from a 3D imager includes (i) 3D spatial information and (ii) associated color information; normalize the color information associated with the first 3D image into a common color space; determining voxels from the 3D spatial information (note: plurality of 3d points, depth data; see claim 10 rejection), and determining histograms from the voxels in the first 3D image.
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to apply the similar process to the second 3D image (model image) obtained as taught by Schulte including: obtain a second 3D image of a model object, the second 3D image including (i) 3D spatial information of the model object and (ii) color information associated with the model object; normalize the color information associated with the model object into a common color space; determine voxels of the second 3D image from the 3D spatial information of the model object; and determine the model histograms from the voxels of the second 3D image and the normalized color information associated with the model object to have the 3D images (first and second images) produce same data structures for efficient comparison processing.
Regarding claim 17, depending on claim 10,
The system of claim 10, wherein the machine readable instructions further cause the system to determine 3D information pertaining to one or more surfaces of the object in the first 3D image (Vaidya: ¶8, "Illustratively, a convexity process measures a degree of a convex shape along at least one surface of the object. The convexity process is constructed and arranged to determine a bulge in height along the at least one surface of the object. Additionally, the refinement process includes a height from bulginess process that refines the height dimension based on a bulginess estimate for the object. The convexity process can be constructed and arranged to (a) fit a plane with respect to boundary edges in the 3D image of the object that correspond to the (e.g.) top surface, (b) obtain a tallest point on the top surface, (c) obtain a tallest point on the boundary edges, and (d) determine a measure of convexity of the top surface using the relative tallest points. The refinement module can be constructed and arranged to adjust the improved cuboidal dimensions based on the determined convexity. Illustratively, the at least one surface is a top surface").
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Schulte and implement Vaidya's teaching wherein the machine readable instructions further cause the system to determine 3D information pertaining to one or more surfaces of the object in the first 3D image to accurately measure the length, width and height of a typically cuboidal object in the field of view in the presence of noise (Vaidya: ¶7).
Regarding claim 18, depending on claim 10,
The combination of Bulzacki, Vaidya, and Schulte teaches:
The system of claim 10, wherein the 3D imager comprises at least one of a time of flight camera, stereo vision camera, structured light camera, a range camera, a 3D profile sensor, or a triangulation 3D imager (Vaidya: ¶52, "The camera 110 can be any assembly that acquires 3D images of objects including, but not limited to, stereo cameras, time-of-flight cameras, LiDAR, ultrasonic range-finding cameras and laser-displacement sensors. A single camera or array of cameras can be provided and the terms “camera” and/or “camera assembly” can refer to one or more cameras that acquire image(s) in a manner that generates 3D image data for the scene"; also see Examiner claim interpretation in the claim objection section).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Schulte and implement Vaidya's teaching wherein the 3D imager comprises at least one of a time of flight camera, stereo vision camera, structured light camera, a range camera, a 3D profile sensor, an a triangulation 3D imager to accurately measure the length, width and height of a typically cuboidal object in the field of view in the presence of noise (Vaidya: ¶7).
Regarding method claims 1-3, and 6-8,
method claims 1-3, and 6-8 are drawn to the methods corresponding to the operations of using same as claimed in apparatus claim 10-12, and 15-17 respectively. Therefore, method claims 1-3, and 6-8 correspond to the operations in the apparatus of claims 10-12, and 15-17 respectively, and is rejected for the same reasons of obviousness as used above.
Regarding CRM claim 19,
CRM claim 19 is drawn to the CRM corresponding to the operations of using same as claimed in the apparatus of claim 10. Therefore, CRM claim 19 corresponds to the operations in the apparatus of claim 10, and is rejected for the same reasons of obviousness as used above.
Claims 4-5, 9, and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Bulzacki in view of Vaidya and Schulte further in view of Young et al. (US 9600929 B1, hereinafter “Young”).
Regarding claim 13, depending on claim 10,
The combination of Bulzacki, Vaidya, and Schulte teaches:
The system of claim 10, and comparison between the first 3d image and a (model image) reference template as discussed in claim 10 rejection.
However, the combination of Bulzacki, Vaidya, and Schulte does not teach the analogous art Young teaches.
Young teaches:
identify sets of corresponding voxels between the voxels of the first 3D image (Young: col. 4 lines 3-8, first set of 3D voxel data), and voxels of the model image (Young: col. 4 lines 3-8, "The first step aligns the first set of 3D voxel data and the second set of 3D voxel data. The second step makes a rapid one-to-one comparison between each voxel in the first set of 3D voxel data and a corresponding voxel in the second set of 3D voxel data to create a diff model that records differences found between the first and second sets of 3D voxel data).
NOTE 13A: It would have been obvious to a PHOSITA to use a model image (reference template) as the second set as taught by Young with corresponding voxel data. As discussed above, the combination of Bulzacki, Vaidya, and Schulte already teaches comparison between the first 3D image and a reference template (model image). In reference to Young col. 2 lines 64-67 to col. 3 lines 1-9, Young describes a 3D voxel model compared to an actual machine (model image).
determine a comparison score for each set of corresponding voxels; and determine the color score as a weighted sum of the comparison scores (note: the comparison score being the similarity/dissimilarity value and the comparison score is the result after a the comparison is made between the total color values or the average color values of the two sets of 3D voxel data as described Young: col. 16 lines 4-35, "Various other methods of calculating a similarity/dissimilarity value of colors between the two sets of 3D voxel data are possible. For example, instead of the ratio of the number of voxels having the same color relative to the total number of voxels in the first set of 3D voxel data, a similarity/dissimilarity value may be calculated based on a comparison between the total or average color values of the two sets of 3D voxel data. If one of the color values being compared is “N/A” meaning that it is associated with an empty voxel, such comparison has little meaning and can be omitted. In the example of FIG. 6C, comparison between “B” and “N/A” and comparison between “N/A” and “R” can thus be omitted. The routine then compares a combination of “R” and “G” in the first set against a combination of “R” and “L” in the second set. The comparison can be made between the total color values or the average color values of the two sets of 3D voxel data (or two subsets of 3D voxel data). In the illustrated example, “R” expressed in RGB decimal code is (255, 0, 0), “G” is (0, 128, 0) and “L” is (0, 255, 0). Thus, the total color value of “R” and “G” in the first set can be calculated as (255, 0, 0)+(0, 128, 0)=(255, 128, 0), and the average color value of “R” and “G” can be calculated as (255, 128, 0)/2=(128, 64, 0). Similarly, the total color value of “R” and “L” in the second set can be calculated as (255, 0, 0)+(0, 255, 0)=(255, 255, 0), and the average color value of “R” and “L” can be calculated as (255, 255, 0)/2=(128, 128, 0). In some embodiments, threshold value(s) may be set to ignore small differences in color values as noise, and not to treat them as actual color differences. For example, while “R” in RGB decimal code is (255, 0, 0), a threshold range of +/−5 may be set such that (250, 5, 5), (254, 1, 1), etc., will all be considered “R” in some embodiments")
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Vaidya, Schulte, and Young to include: identify sets of corresponding voxels between the voxels of the first 3D image, and voxels of the model image; determine a comparison score for each set of corresponding voxels; and determine the color score as a weighted sum of the comparison scores and implement Young's teachings to determine and visualize differences between the 3D models (Young: col. 1 lines 10-11).
Regarding claim 14, depending on claim 13,
The combination of Bulzacki, Vaidya, Schulte, and Young teaches:
The system of claim 13, wherein to identify sets of corresponding voxels, the machine readable instructions further cause the system to perform a 3D transformation on at least one of the first color image or the model image (Young: col. 4 lines 28-44, "According to another aspect of the invention, a non-transitory computer-readable storage medium is provided, which includes computer-executable instructions for executing a diffing routine including generally three steps. The first step aligns a first set of 3D voxel data of an object and a second set of 3D voxel data by matching a level of detail (LOD) of the first set and an LOD of the second set. For example, a LOD may be defined as an actual dimension of an object that is 3D-modeled divided by a number of voxels to which the actual dimension is mapped (e.g., 3-inch long object mapped to 9 voxels; LOD=3/9=1/3). The second step makes a one-to-one comparison between each voxel in the first set of 3D voxel data and a corresponding voxel in the second set of 3D voxel data to create a diff model that records differences found between the first and second sets of 3D voxel data. The third step displays the content of the diff model on a display")
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Bulzacki, Vaidya, Schulte, and Young to include: wherein to identify sets of corresponding voxels, the machine readable instructions further cause the system to perform a 3D transformation on at least one of the first color image or the model image. and implement Young's teachings to determine and visualize differences between the 3D models (Young: col. 1 lines 10-11).
Regarding claim 9, depending on claim 1,
The combination of Bulzacki, Vaidya, Schulte, and Young teaches:
The method of claim 1, further comprising determining, via the processor, one or more normal vectors of the surface of the object in the first 3D image (Young: col. 11 lines 40-46, ("For example, each of the 4 voxels in FIG. 3B is associated with occupancy information (value 1 indicating an occupied voxel and value 0 indicating an empty voxel) and attributes information such as an RGB color value, a normal vector (which indicates a direction of illumination to define shine), an intensity value (which defines brightness and darkness), weight, density, temperature, etc. of each occupied voxel"; Young: col. 1 lines 27-56, “As described above, a 3D scanning/sampling/modeling device may generate 3D digital data by collecting a complete set of (x, y, z) locations that represent the shape of an object. The term “point cloud” is used to describe such point sets including a large number of points. The data points in point cloud data are unorganized and spaced irregularly within the cloud. While in some applications point cloud data can directly be used to generate 3D images, in other applications point cloud data can be transformed to volume graphics or 3D voxel data, which can then be used to generate 3D images. Some exemplary systems and methods for transforming point cloud data to volume graphics data are disclosed in commonly owned U.S. Pat. Nos. 7,317,456 B1 and 7,420,555 B1 both entitled “METHOD AND APPARATUS FOR TRANSFORMING POINT CLOUD DATA TO VOLUMETRIC DATA,” which are incorporated herein by reference. In volume graphics, volume elements (i.e., “voxels”) are the base data used to represent 3D objects. Typically, voxels are simply pixels that have a third coordinate z in addition to x and y coordinates in a Cartesian coordinate system (though voxels may take various other forms, also). In other words, voxels are equally sized cubes that form a discretely defined 3D space.
Volume graphics represent a promising way to achieve the degree of realism required for high quality 3D simulations and visualization applications because volume models can contain all the surface and internal characteristics of a real object. This is in contrast to, for example, polygon-based graphics, in which a mesh of polygons is used to represent only the surfaces of a 3D object”).
Regarding method claims 4-5,
method claims 4-5 are drawn to the methods corresponding to the operations of using same as claimed in apparatus claim 13-14 respectively. Therefore, method claims 4-5 correspond to the operations in the apparatus of claims 13-14 respectively, and is rejected for the same reasons of obviousness as used above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK GALERA whose telephone number is (571)272-5070. The examiner can normally be reached Mon-Fri 0800-1700 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK P GALERA/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights